kue

I have never used Kafka or had to maintain queues with more than a hundred jobs per hour. Most of the jobs queues I've had to maintain are doing simple things like sending emails or updating a third party API. I've always resorted to simpler distributed redis based job queues such as kue [0], bull [1], celery [2] or resque [3]. I would like to learn more about distributed queues and architectures from first principles. Where is a good place to go? [0] https://github.com/Automattic/kue [1] https://github.com/OptimalBits/bull [2] http://www.celeryproject.org/ [3] https://github.com/resque/resque

Regarding #2, the Ruby sensibility on pulling in dependencies is very different than the Node world's. As an arbitrary example, consider the deps of two similar packages, delayed_job (Ruby, https://github.com/collectiveidea/delayed_job) and Kue (JS, https://github.com/Automattic/kue). Delayed_job has two dependencies if you're running on Ruby (rather than JRuby): rake and sqlite3. Neither rake nor sqlite3 have any dependencies of their own - in production mode, of course. On the other hand, Kue has nine direct dependencies, each of which have their own. The full dependency tree of Kue has one hundred and eighty two separate dependencies.

Job queuing kue.js https://github.com/Automattic/kue

Hi, I'm the author of this project. The code will be available soon on github: at https://github.com/valentinvieriu/visual-hacker-news - right now that's the old version using Angularjs The project uses as a starting point the amazing Vuejs Implementation from https://github.com/vuejs/vue-hackernews-2.0 . It has super fast server side rendering, that's why the feedback is imediate as you load the page. The thumbnails are generated using phantomjs server. I'm using https://github.com/blockai/phantom-pool to make sure the phantomjs instance is reused. This improves the speed of the screenshots quite well. Also I'm using a scheduling node package (https://github.com/Automattic/kue) to make sure that images are pre cached as the link are submited to hackernews. I'll post the code on https://github.com/valentinvieriu/vue-hackernews-2.0, but n

The thumbnails are generated using phantomjs server. I'm using https://github.com/blockai/phantom-pool to make sure the phantomjs instance is reused. Also I'm using a scheduling node package (https://github.com/Automattic/kue) to make sure that images are precached as the link are submited to hackernews. I'll post the code on https://github.com/valentinvieriu/vue-hackernews-2.0, but need to take care of some hardcoded redis credentials first. I'll give a reply when it's up

we've used redis for our scheduled jobs and does a pretty good job. we still do use mongoDB for our 'main' data. the reason for the redis option was because we liked the 'kue' - https://github.com/learnboost/kue

I thought the same. For redis there's Kue, which looks a lot more feature complete. https://github.com/learnboost/kue

Cool! I haven't heard of Kue. I'll take a look! Source for the curious: https://github.com/learnboost/kue