que

Oban is fantastic and has been a huge source of inspiration for us, showing what is possible in this space. In fact I think during my time at Distru we were one of Parker's first customers with Oban Web / Pro :) We've also had a lot of experience with with other libraries like Que ( https://github.com/que-rb/que ) and Sidekiq (https://sidekiq.org/) which have certainly influenced us over the years.

> Can you define "low throughput"? IDK maybe <1000 messages per minute Not saying SKIP LOCKED can't work with that many. But you'll probably want to do something with lower overhead. FWIW, Que uses advisory locks [1] [1] https://github.com/que-rb/que

slower to run, but when you keep the postgres connection open you will know that the job is still running, while with for update skip locked you would need to have a status and a job_timeout basically. so pg_try_advisory_lock/pg_advisory_unlock can lock over transactions while for update skip locked can't, thus you would either need to keep a transaction open or use status+job_timeout (and in postgres you should not use long transactions) basically we use c#, but we looked into https://github.com/que-rb/que which uses advisory_locks, since our jobs take like 1 min to 2 hours it was a no-brainer to use advisory_locks. it's just not the best thing if you have thousands of fast jobs per second, but for a more moderate queue where you have like 10000 jobs per minute/10 minutes/30 minutes and they take like 1 min to 2 hours its fine. we also d

Few things. 1. The main downside to using PostgreSQL as a pub/sub bus with LISTEN/NOTIFY is that LISTEN is a session feature, making it incompatible with statement level connection pooling. 2. If you are going to do this use advisory locks [0]. Other forms of explicit locking put more pressure on the database while advisory locks are deliberately very lightweight. My favorite example implementation is que [1] which is ported to several languages. [0] https://www.postgresql.org/docs/current/explicit-locking.htm... [1] https://github.com/que-rb/que

(not sure why this comment was dead, I vouched for it) There are a lot of ways to implement a queue in an RDBMS and a lot of those ways are naive to locking behavior. That said, with PostgreSQL specifically, there are some techniques that result in an efficient queue without locking problems. The article doesn't really talk about their implementation so we can't know what they did, but one open source example is Que[1]. Que uses a combination of advisory locking rather than row-level locks and notification channels to great effect, as you can read in the README. [1]: https://github.com/que-rb/que