exhibitor

ASGs + Zookeeper is really easy, but you have to use https://github.com/soabase/exhibitor. It coordinates replacing downed ZK nodes using S3, and also provides a bunch of extra management tools. I had a single Zookeeper ASG running under load for over three years without maintenance. I pinged my old co-founder to see if he's willing to open source the CloudFormation template.

People talk about Zookeeper as a negative, but in a quorum it's been one of the most stable and reliable pieces of software I've deployed, despite being a bit frustrating to set up / configure. Netflix's Exhibitor[1] is an indispensable addition to it. Also, once you're on the Big Data™ train, a lot of things like to plug into Zookeeper, so it becomes more of a convenience. Kafka, and presumably DL, are at their most useful when you're pushing the limits of NIC and/or HDD performance for throughput. Zookeeper's configuration is a footnote in the complexity of managing one of these systems, and lets them avoid implementing their own byzantine coordination system. Also, folks seem to appreciate Aphyr's opinion, and he states it pretty plainly: Use Zookeeper. It’s mature, well-designed, and battle-tested. [2] [1] https://github.com/Netflix/e

I use ZooKeeper in production for snitch.io. There are some interesting new alternatives such as etcd / serf/consul - but at the time ZooKeeper had the best track record (under Jepsen analysis). Things might have changed since then. Aphyr has done a bunch of analysis of these systems part of his Jepsen tool: http://aphyr.com/tags/jepsen and http://aphyr.com/posts/291-call-me-maybe-zookeepe If you are going to use ZooKeeper I strongly suggest looking at both Apache Curator and Netflix Exhibitor (they are complimentary). The examples bundled with ZK don't handle all errors/edge cases... Curator is a library of common patterns available to use mostly out of the box. Exhibitor is a ZooKeeper "aware" supervisor system: https://github.com/Netflix/exhibitor Also always remember your ensemble should have an odd number of nodes (3,5,7)

No at all clear to me. You think https://github.com/Netflix/exhibitor/blob/master/gradle/rele... is clear? What about https://github.com/Netflix/exhibitor/blob/master/gradle/lice...? What is "ext.year"? I don't want to learn and understand an entirely new language for my build. I am not able to maintain these build scripts. This is the problem with DSLs in general. Maven isn't great, but its strength is standardization. I can look at any Maven pom and know what it does and make changes to it.

Here's Netflix Exhibitor (which I wrote): https://github.com/Netflix/exhibitor - the developer who wrote the build scripts did an incredible job. I'm very impressed. However, I don't comprehend it. Note: this was previously in Maven and had exactly 3 POM files.