pgqueuer

PgQueuer is a Python job queue that uses PostgreSQL for everything — queue storage, locking, notifications. No Redis, no RabbitMQ, no separate broker to operate. If your app already has Postgres, you already have the broker. After ~2 years and 26 minor releases, 1.0 ships today as the first stable release. Strict semver from here on. Thanks to everyone who contributed and left feedback!

PgQueuer turns PostgreSQL into a background job queue using LISTEN/NOTIFY and FOR UPDATE SKIP LOCKED. v0.26.0 adds an in-memory adapter so you can run the same code in tests and CI without a database. Internally it's now built on port protocols, so you can swap the persistence layer entirely. EDIT; HN trims the full url to readme about in-memory https://github.com/janbjorge/pgqueuer?tab=readme-ov-file#in-...

I maintain a small Postgres-native job queue for Python called PGQueuer: https://github.com/janbjorge/pgqueuer It uses the same core primitives people are discussing here (FOR UPDATE SKIP LOCKED for claiming work; LISTEN/NOTIFY to wake workers), plus priorities, scheduled jobs, retries, heartbeats/visibility timeouts, and SQL-friendly observability. If you’re already on Postgres and want a pragmatic “just use Postgres” queue, it might be a useful reference / drop-in.

If you like the “use Postgres until it breaks” approach, there’s a middle ground between hand-rolling and running Kafka/Redis/Rabbit: PGQueuer. PGQueuer is a small Python library that turns Postgres into a durable job queue using the same primitives discussed here — `FOR UPDATE SKIP LOCKED` for safe concurrent dequeue and `LISTEN/NOTIFY` to wake workers without tight polling. It’s for background jobs (not a Kafka replacement), and it shines when your app already depends on Postgres. Nice-to-haves without extra infra: per-entrypoint concurrency limits, retries/backoff, scheduling (cron-like), graceful shutdown, simple CLI install/migrations. If/when you truly outgrow it, you can move to Kafka with a clearer picture of your needs. Repo: https://github.com/janbjorge/pgqueuer Disclosure: I maintain PGQueuer.

pgqueuer turns vanilla PostgreSQL into horizontally-scalable job queue with zero extra infrastructure. If you’re running Postgres and want to ditch extra queue clusters, give it a spin and let me know how it holds up. https://github.com/janbjorge/pgqueuer

Hi HN, I’m the maintainer of PGQueuer, a minimalist job queue for Python that runs entirely on PostgreSQL—no Redis, brokers, or additional infrastructure required. It’s designed to offer solid concurrency with low overhead and fit seamlessly into projects that already use Postgres. I’ve been working on improving the public documentation [1], and I’m looking for feedback from others who’ve gone through this process—especially those maintaining small, technical open-source projects where all the important knowledge lives in the docs. There’s no internal documentation and likely never will be, so clarity and accessibility are essential. I’d especially appreciate your thoughts on: * How do you decide what’s worth documenting for a technical audience who might just read the source anyway? * How do you balance being thorough without overwhelmin

PGQueuer does something fairly similar in python: https://github.com/janbjorge/pgqueuer

Another good one is pgqueuer https://github.com/janbjorge/pgqueuer

PGQueuer is a lightweight job queue for Python, built entirely on PostgreSQL. It uses SKIP LOCKED for efficient and safe job processing, with a minimalist design that keeps things simple and performant. If you’re already using Postgres and want a Python-native way to manage background jobs without adding extra infrastructure, PGQueuer might be worth a look: GitHub - https://github.com/janbjorge/pgqueuer

PGQueuer provides job queuing, locking, and real-time notifications using PostgreSQL's FOR UPDATE SKIP LOCKED and LISTEN/NOTIFY. It's a minimalist alternative for those already using PostgreSQL, avoiding the need for Redis. - https://github.com/janbjorge/PGQueuer disclaimer: Im the auther.

Unsure whats going on, but unable to add links to my question. - https://github.com/janbjorge/pgqueuer - https://news.ycombinator.com/item?id=41284703

Hi everyone! I’m currently enhancing the documentation for my project, PGQueuer. About PGQueuer: PGQueuer is a minimalist, high-performance job queue library for Python, leveraging PostgreSQL’s LISTEN/NOTIFY for efficient job management. It’s perfect for handling background tasks and managing workflows with simplicity and reliability https://github.com/janbjorge/pgqueuer

At first glans i i see two thins that PgQueuer can. It has native async support, it you can also have sync functions, they will be offloaded to threads via anyio (https://github.com/janbjorge/PgQueuer/blob/99c82c2d661b2ddfc...) It has a global rate limit, synced via pg notify.