HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
etcd-io
etcd
Go2013–2024
etcd-io / etcd

etcd

Distributed reliable key-value store for the most critical data of a distributed system

Data & DatabasesSystems & InfrastructureDevOps & Cloud
Explore on GitHubRead Hacker News discussions
79Distinct people
108Mentions
0Explicit recommenders
52.6KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Distributed reliable key-value store for the most critical data of a distributed system

GitHub

Where it is used

  • Fidelity Investments — “We use CoreDNS for service discovery, etcd as KV store, Fluentd for logging, Helm as a package manager, Kubernetes for container orchestration, rely on CNI …” Source

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

Mentioned in the same Hacker News threads. That is not proof they are substitutes.

This week on Hacker News

Nov 8–Nov 15 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/15/2026, 8:27:21 AM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year108 dated mentions

Observed mentions: 2013-07-31 15:48:56 UTC to 2024-11-28 01:23:10 UTC.

051015202013-01-01 00:00:00 UTC to 2014-01-01 00:00:00 UTC (exclusive): 11 mentions; 2 submissions, 9 comments20132014-01-01 00:00:00 UTC to 2015-01-01 00:00:00 UTC (exclusive): 13 mentions; 0 submissions, 13 comments2015-01-01 00:00:00 UTC to 2016-01-01 00:00:00 UTC (exclusive): 20 mentions; 2 submissions, 18 comments2016-01-01 00:00:00 UTC to 2017-01-01 00:00:00 UTC (exclusive): 15 mentions; 0 submissions, 15 comments2017-01-01 00:00:00 UTC to 2018-01-01 00:00:00 UTC (exclusive): 12 mentions; 0 submissions, 12 comments2018-01-01 00:00:00 UTC to 2019-01-01 00:00:00 UTC (exclusive): 8 mentions; 0 submissions, 8 comments20182019-01-01 00:00:00 UTC to 2020-01-01 00:00:00 UTC (exclusive): 6 mentions; 1 submissions, 5 comments2020-01-01 00:00:00 UTC to 2021-01-01 00:00:00 UTC (exclusive): 8 mentions; 0 submissions, 8 comments2021-01-01 00:00:00 UTC to 2022-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2022-01-01 00:00:00 UTC to 2023-01-01 00:00:00 UTC (exclusive): 4 mentions; 2 submissions, 2 comments2023-01-01 00:00:00 UTC to 2024-01-01 00:00:00 UTC (exclusive): 3 mentions; 0 submissions, 3 comments2024-01-01 00:00:00 UTC to 2025-01-01 00:00:00 UTC (exclusive): 7 mentions; 0 submissions, 7 comments2024

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
20132013-01-01 00:00:00 UTC2014-01-01 00:00:00 UTC1129
20142014-01-01 00:00:00 UTC2015-01-01 00:00:00 UTC13013
20152015-01-01 00:00:00 UTC2016-01-01 00:00:00 UTC20218
20162016-01-01 00:00:00 UTC2017-01-01 00:00:00 UTC15015
20172017-01-01 00:00:00 UTC2018-01-01 00:00:00 UTC12012
20182018-01-01 00:00:00 UTC2019-01-01 00:00:00 UTC808
20192019-01-01 00:00:00 UTC2020-01-01 00:00:00 UTC615
20202020-01-01 00:00:00 UTC2021-01-01 00:00:00 UTC808
20212021-01-01 00:00:00 UTC2022-01-01 00:00:00 UTC101
20222022-01-01 00:00:00 UTC2023-01-01 00:00:00 UTC422
20232023-01-01 00:00:00 UTC2024-01-01 00:00:00 UTC303
20242024-01-01 00:00:00 UTC2025-01-01 00:00:00 UTC707

What Hacker News says

20 sources
philips·Nov 5, 2015
This is some interesting behavior from the blog post: Eventually, Python exits, and closes TCP connection A, by sending a FIN packet – but etcd doesn’t send a FIN of its own, or send a zero-byte chunk to finish the 200 OK, as it does in the success case. This causes the socket to leak on etcd’s side – we think this is a bug in Go’s HTTP stack. This really should work so we need to investigate that one more deeply. I am not opposed to adding the timeout option as the author would like to do and we would happily consider a patch for this. Update: here is a proof-of-concept patch to address this feature request: https://github.com/coreos/etcd/pull/3816 Longer term etcd will use gRPC/http 2.0 in the new etcd v3 API[1] to address use cases like this. In particular gRPC will allow us to have multiple streams while kee
Read the rest
ping open a single TLS connection. A major expense of the current watch API is having to tear down connections as explained in this post. We will still support http/rest likely through the use of grpc-gateway[2]. The file descriptor rlimit issue that was the side-effect of this issues is why databases like etcd have to protect themselves. For example a few months ago etcd learned to reserve file descriptors for itself in case we get clients hogging up resources[2]. Longer term we would like to be able to let proxies handle watches directly to further protect the core quorum of etcd. [1] https://github.com/coreos/etcd/blob/master/Documentation/rfc... [2] https://github.com/gengo/grpc-gateway [3] https://github.com/coreos/etcd/pull/3219
In Adventures in debugging: etcd, HTTP pipelining, and file descriptor leaks
philips·Sep 11, 2015
On the stability/reliability of etcd the team is very confident in how it behaves. Here is how we build that confidence: - We have a functional testing cluster that is constantly simulating various types of failures. You can find some information on that here: https://coreos.com/blog/new-functional-testing-in-etcd/ - The team is works to ensure the raft library that we use (and that we now share with cockroachdb) has unit tests that handle all of the edge cases described in the raft paper. See this paper as an example: https://github.com/coreos/etcd/blob/master/raft/raft_paper_t... - We have been running etcd in production for 1.5 years for a bootstrapping service called discovery.etcd.io and have focused on how to handle higher and higher read/write loads there. For example see the improvements we made in etcd 2.1 with async snapshots: h
Read the rest
ttps://twitter.com/BrandonPhilips/status/639843455141740544 - We have also focused on clear guides on how to operate etcd. All of this testing and performance work is of no use if people can't operate the system correctly: https://coreos.com/etcd/docs/latest/ Of course if you have particular issues that you have encountered we would love to help: https://github.com/coreos/etcd#contact
In Etcd 2.2
merb·Jul 17, 2015
It simply relies on the Voting feature of ETCD (Raft) it's really simple to use locking with etcd, and etcd is really really stable. However it would be easier to install etcd on every Postgres node and just make a golang library that sets the master of Postgres to the etcd master (etcd also has a leader). Also systemd would keep the overall system healthy. (that's what we at envisia do) Just have repeatedly check if the machine is the leader and if yes it sets the url of the currently running machine to a etcd key. So overall we need to use 3 Postgres machines and 1 could fail and we would still have voting, however thats just for a single master where we don't need to read from the slaves, however thats easily extendable. Oh and here is the Compare and Swap (Atomic) functionality of etcd that he described: https://github.com/coreos/etcd
Read the rest
/blob/master/Documentation/api...
In High Availability for PostgreSQL, Batteries Not Included
mackwic·Jun 14, 2015
> Tectonic is a commercial distribution of the combined Kubernetes and CoreOS stack. > Tectonic pre-packages all of the open source components required to build a Google-style infrastructure and adds additional commercial features, such as a management console, corporate SSO integration, and Quay, our enterprise ready container registry. Well, I don't care. Fix etcd before anything. Cluster recovery doesn't work well (it's an understatement), etcd has issues with high loads, and leader election has its quirks. CoreOs is barely usable at this state. This is the kind of thing you need to expect when running a CoreOS cluster: https://github.com/coreos/etcd/issues/863. etcd is not "production ready", well, this should be taken care of as it's a central part of the system.
In Introducing Tectonic – A Kubernetes and CoreOS Platform
joshuak·May 28, 2015
I use etcd[1] which is an implementation of Raft for cluster configuration. I have a hard time seeing how Raft or consensus in general scales to a general solution for reliable consistency services. One can not scale by adding servers because all potential members must be known, and consistency is only guaranteed if you are willing to require confirmations forcing full round trip communication with all members for every write. Anything more than 7 servers or so, and things start getting dodgy. Try to improve reliability by adding servers in geographically diverse areas is untenable altogether. I don't see how simply having multiple servers with failover, and a version clock isn't equally or even more reliable due to it's scalability. The byzantine generals problem[2] is more about reliable voting and trust rather that high performance
Read the rest
and reliability. I'd very much appreciate more insight on this. 1. https://github.com/coreos/etcd 2. http://en.wikipedia.org/wiki/Byzantine_fault_tolerance
In Raft Consensus Algorithm
iyn·May 4, 2015
Do you mean the differences to the way I was setting up servers before CoreOS? Well, this is subjective, of course, but I feel that it's easier & faster to isolate everything now. I still have to add iptables rules (I also use AWS ports restrictions) and add SSH keys, but that's all I have to do. Auto updates with scheduled reboots and safe rollback work out of the box, I have only few vector attacks in terms of services exposed to the public network. When I have everything in 1 data center, "internal" tools (etcd, fleet, confd, locksmithd) are open only in private network and just ports 22, 80 & 443 have to be exposed. Containers are also "linked" in private network, I can easily securely connect services/applications in containers running on different machines without much overhead. Often the easiest attack vector is/are security vulner
Read the rest
abilities in applications exposed to the public network. Containers are great here, because when application is compromised, just 1 particular container is dead (and probably other containers running that app). I can just run docker kill app && docker rm app and the rest of my containers are (probably) ok. The problem is that attacker can gain access to data in etcd, since it's not encrypted by default and has no per-user permission (but you can use HTTPS in etcd cluster, which is good), however as of now you can use something like crypt: https://github.com/xordataexchange/crypt to use gpg in etcd (with natural API). Docker gives you root for everything in the container, which may not be the best option. Also, only lately Docker have the option to verify downloaded images. rkt already has this feature, and it's just 0.5.5. (see also: https://github.com/coreos/etcd/blob/master/Documentation/sec...). Docker is great, don't get me wrong, I think it's a very good software and I'm glad I can use it. CoreOS team has just a better vision and priorities, IMO. I'm considering a blog post about CoreOS/Docker/rkt and similar tools, not sure if there's interest?
In Kubernetes is going to support rkt
davismwfl·May 2, 2015
This isn't a new problem and unless you have a compelling reason to write your own, I'd look at existing open source alternatives like zookeeper https://zookeeper.apache.org/ or etcd https://github.com/coreos/etcd. Each has its own plus/minuses We started ours as just a config document stored in the NoSQL db, but found using a service like etcd or zookeeper is way better. This is especially true as the number of services increases and the complexity of configuration goes up.
In Ask HN: SOA/Microservices central config systems?

Showing the 15 most recent of 20 sources. Enable JavaScript to search and browse every indexed mention.

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.