raft

Litestream author here. The default configuration is to replicate any new changes every second to S3. This is mainly a cost issue though. S3 doesn't charge for ingress bandwidth but it does charge $0.000005 per PUT request. If you have a non-stop, continuous stream of writes against your database then that's about $12/mo. In practice, most people don't have a continuous stream of writes. Michael Lynch, for example, wrote a post about how he pays $0.03/month for his Litestream replication[1]. I've worked with Raft a lot in my career. I wrote the original etcd Raft implementation (sorry!) [2] and even wrote a visualization explaining Raft[3]. It's a great tool when you need really high durability guarantees but I'm not convinced it gives you better availability guarantees than a single node that can quickly be recovered. [1]: https://mtlync

Hello -- rqlite author here. It was definitely interesting to code this. It was 50% to meet a need, 50% to better understand Raft consensus in general, and Ben Johnson's go-raft implementation in particular (https://github.com/goraft/raft). As it happens Ben and I now work together, on InfluxDB (http://influxdb.com/), which has a new Raft implementation at its core. Definitely fun.

For the split vote see https://github.com/goraft/raft/issues/13 The conflicting data under partition is covered I think - the set that doesn't meet quorum won't accept writes and will see it has a lower election term than the other partition when it re-joins. EDIT: From a use vase point of view, it simplifies the construction of CP systems. This has directly led to etcd and consul, which would be many times more complex had their authors had to implement paxos. Both etcd and consul are still young software, but if you take a look at the 'Call me Maybe' series of blog posts it's pretty apparent that there's a massive deficiency in current systems handling of network partitions. http://aphyr.com/posts/281-call-me-maybe-carly-rae-jepsen-an...

By systems programming I mean writing the code that applications and distributed systems run on top of. Raft (https://github.com/goraft/raft) and groupcache (https://github.com/golang/groupcache) come to mind as examples. That's a good point though. A lot of people mean different things by systems programming.

The consistent key-value store portion is built using a new implementation of Raft (https://github.com/hashicorp/raft) as opposed to the well-known goraft (https://github.com/goraft/raft). Would be interesting to compare and contrast the two!

The underlying Raft library for etcd can be found over here: https://github.com/goraft/raft Some additional resources on Raft, including a link to the raft-dev mailing list, can be found over here: http://raftconsensus.github.io/ Raft has worked really well on etcd so far and it is nice to have a simple easy to reason about algorithm to work on top of. Etcd is moving quickly adding a simple to use fair mutex API, improved scriptability via etcdctl and several improvements to the core API in recent weeks. Come join in.

Here's the raft algorithm we chatted about: https://ramcloud.stanford.edu/wiki/download/attachments/1137... And an implementation: https://github.com/goraft/raft Decentralised conversations :)

I got interested in building something like this when I saw Docker issue 410[1]. If my understanding is correct Mesos uses Zookeeper to keep a Mesos master available. Zab, Zookeeper's distributed consensus algorithm, is similar to Raft which has a popular Go implementation[2] written by Ben Johnson. Perhaps Flynn agents can use this implementation. 1. https://github.com/dotcloud/docker/issues/410 2. https://github.com/benbjohnson/go-raft

I did a Go implementation of the Raft distributed consensus protocol. The protocol works similarly to Google's Chubby, Apache Zookeeper and Heroku's doozerd except that it works as an embeddable library and not as a separate server. There's also a reference implementation called raftd. https://github.com/benbjohnson/go-raft https://github.com/benbjohnson/raftd