supavisor

Supabase | Senior Elixir Engineer | APAC | Remote | https://supabase.com/ We're hiring to grow our Elixir teams mainly on Supavisor and Realtime projects. You will be using Elixir and also help build new client libs mainly in TS. We work in an opensource manner so you can check our code in https://github.com/supabase/realtime and https://github.com/supabase/supavisor. * Senior Elixir Engineer: https://jobs.ashbyhq.com/supabase/46ffc00b-b9a7-4b45-a6f2-1a... Any question, email me at filipe@supabase.io

WalEx instead of pub/sub (listen/subscribe): https://github.com/cpursley/walex Supavisor connection pooler: https://github.com/supabase/supavisor

[I'm on the supabase team] You can find the code/docs here: https://github.com/supabase/supavisor This release adds support for - SQL Parsing - Load balancing - support for named prepared statement - query cancellation It's also now available on all new databases in Supabase. For some more background on scalability, we have some benchmarks available here: https://supabase.com/blog/supavisor-1-million

PgBouncer maintainer here, so obviously biased. But I think currently PgBouncer should still be the default connection pooler that you choose. There's a few newer options: Odyssey, pgcat, and supavisor. But all focus on a solving 1 or 2 specific problems that PgBouncer did not solve well, while not solving many of the other problems that PgBouncer does solve. So if you have the exact same requirements as the authors of those tools, then switching might be good. But otherwise you should probably continue using PgBouncer. Supavisor specifically is too immature for general usage IMHO. It's missing some really core functionality like query cancellations: https://github.com/supabase/supavisor/issues/174 Also it's now the only connection pooler without prepared statement support. I did a talk on this exact topic at PGConf NYC recently. My slide

What's the path forward for PgBouncer now that Supabase released their version written in Elixir? https://github.com/supabase/supavisor

Phoenix.PubSub is basically a noop service. It really just works. If discovering nodes is difficult in your env, try using a listen/notify libcluster strategy: https://github.com/supabase/supavisor/blob/main/lib/cluster/...

I found a relevant GitHub issue for this: https://github.com/supabase/supavisor/issues/85

Have you contacted the Supavisor team about adding listen/notify support? Supavisor may be your way forward: https://github.com/supabase/supavisor

Connection poolers like PgBouncer [1] (traditional) and Supabase's Supavisor [2] (new) come to mind. [1] https://www.pgbouncer.org/ [2] https://github.com/supabase/supavisor

For sure > https://github.com/supabase/supavisor/issues/69

It is very BEAMy because we love the BEAM. All connections are supervised :D Might be a little too close to "supervisor" though. > To what extent are these metrics distorted by use of a Supavisor connection pooler? How could Supavisor compensate for that? You mean if you have an Elixir app and using Supavisor for connection pooling? They would be affected as much as they would be if you used PgBouncer. For the metrics Supavisor is exposing, it has those Ecto metrics for the interactions with the management database. Supavisor doesn't use any libs for handling the wire protocol. It handles the wire protocol directly. We are exposing some similar ones: https://github.com/supabase/supavisor/wiki/Metrics

We discussed this. Don't remember exactly but @abc3erl had a lot of experience with poolboy. We made sure to do local benchmarks along the way and there wasn't a meaningful difference. Really the discussion was around doing as little message passing as possible to not copy data. See: https://github.com/supabase/supavisor/pull/4 https://github.com/supabase/supavisor/issues/7 So there are definite gains there but if we want to do things across a cluster we're going to be copying data. The decision at the time was the benefits of clustering outweighed the perf gains considering we were basically already as fast as PgBouncer. This could change.

hey hn, supabase ceo here this is a postgres connection pooler. it’s similar to pgbouncer, but built with Elixir and specifically designed for multi-tenancy. it’s still under development, but it’s at a stage where we can gather a feedback from the community and you can try it yourself. we aren’t using this in production yet, but aiming to deploy it for a subset of databases in the next 2 months. We have the following benchmarks (details in the readme): - Elixir Cluster maintaining 400 connections to a single Postgres database - 1_000_000 clients connecting to the Elixir cluster - Sending 20_000 transactions per second - Consuming 7.8G RAM and ~50% CPU on a 64vCPU machine supavisor can be run as a cluster or a single node/binary. It’s handling 90%+ of the throughput of pgbouncer on a local machine (running pgbench) we will place