nsq

Interesting. What are you thoughts on NSQ? https://github.com/nsqio/nsq Was looking at it earlier today, but haven't ever tried it out.

I know of several startups in NYC using Go to great effect; bitly's nsq comes to mind: https://github.com/bitly/nsq The NY Times open sourced a very cool tool "streamtools" that was built in Go: https://source.opennews.org/en-US/articles/introducing-strea... Your question also reminds me of an interesting talk I saw at RICON: https://speakerdeck.com/al3x/nobody-ever-got-fired-for-picki...

yes - I know what you mean. Part of the reason is that there has been an entire body of effort into making queues scale and be available. Interestingly, just today I discovered a new queue that I did not know of before, written in Go [1] that keeps everything in memory and tries to fail over. I think by the time you need a distributed commit, you have the staffing required to build a queue that scales and does not go down. So there is less focus on semantics of distributed commit (which needs a lot of complexity on the application side and is hard to reason about), than to build a scalable queue. [1] https://github.com/bitly/nsq

Anyone here use NSQ in production (https://github.com/bitly/nsq). If so what are your thoughts in comparison to the article?

disclaimer: co-author of NSQ [1] here Agreed. Message queues play an important role for us (bitly) in being a layer of fault tolerance, buffering, and a means to perform various operational tasks. They're so important to us that we decided to build something that worked exactly like we wanted. NSQ is a realtime distributed message processing system where we've taken the approach of focussing on making it ops friendly and easy to get started. IMO, solutions that make it easy to develop on and administrate are most important... because things break. NSQ is straightforward to deploy (limit dependencies), simple to configure (runtime discovery), and client libraries provide a lot of functionality important for handling failures (like backoff, deferred messages, etc.) for a variety of use cases. We've written an in-depth introductory blog po

The focal point is discovery. Not other queues (or other libraries that can build queues). This is an interesting way to approach it, thanks for open sourcing. We chose to solve the discovery problem a bit differently in NSQ (https://github.com/bitly/nsq) but I could certainly see some interesting opportunities to experiment with a distribute hash table approach as well.

Thanks for the reminder. Getting binary builds up is on our to-do list, so check back soon for those. In the mean time, installing Go is pretty easy (you can use brew, or the official OSX package (assuming you are on OSX) http://golang.org/doc/install) and we've tried to leave clear steps for building NSQ here https://github.com/bitly/nsq/blob/master/INSTALLING.md (there really are very few dependencies other than go itself)