realtime

I'm basically doing the same thing with Hasura, but instead of using their Events webhooks, listening to the WAL for changes with Elixir (I stole the listening code from Supabase realtime: https://github.com/supabase/realtime). I've never written so little backend code for such high performance in so little time. If I were starting today, I'd probably use Supabase for the API as well (which is PostgREST under the hood).

https://github.com/supabase/realtime looks very interesting. Can this be used to replace Debezium & Kafka for syncing Elasticsearch?

Hey HN, Supabase is an open source Firebase alternative. We're building the features of Firebase using enterprise-grade open source tools. We're particularly focused on scalability. We take proven tools like Postgres, and we make them as easy to use as Firebase. Supabase had an Alpha launch [1] six months ago. This time we're more stable and feature-full. For the past 2 months we've run benchmarks [2] and security audits and we're confident to take on production workloads. Our current feature-set includes a realtime engine [3] on top of Postgres, auto-generated APIs using PostgREST [4], and Auth with Row Level Security, OAuth, and magic links [5] . We have a full dashboard, with built-in SQL Editor (full Monaco editor) and a Table View (like Airtable) [6]. Next we are focused Storage & Functions, as well as the local development experienc

> Does scaling a Supabase app basically just boil down to scaling a Postgres database? Correct. We will soon add vertical/horizontal scaling (early next year). For concurrent connections, we use PostgREST[1] for RESTful calls, and an Elixir/Phoenix server [2] for realtime data. This reduces the number of connections to your DB, handling concurrency in the middleware. We are building the benchmarks now, but "enterprise scale" is top of mind with everything we are doing. > Do you have a built in authentication solution? Yes, we use GoTrue [3] for authentication, and we use Postgres Row Level Security for authorization. You can read more about it here: https://supabase.io/blog/2020/08/05/supabase-auth [1] http://postgrest.org/en/v7.0.0/ [2] https://github.com/supabase/realtime [3] https://github.com/netlify/gotrue

It's the same for supabase without the additional graphql layer. It runs a separate elixir project to convert postgres changes into real time subscriptions. You can use this without using other parts. https://github.com/supabase/realtime

Not OP but you will get far with Amazon RDS or another hosted Postgres offering if you can afford it. I'm always terrified of ecomm using NoSQL for carts and transactions but I'd say it's overblown on my part I just prefer to rely on the durability of a SQL database using guardrails at the schema level to keep my data correct. Either tool is easily misused but Postgres really can handle a lot it just doesn't get the shiny attention some other host services do because it doesn't over promise things you probably won't need to use. That said, you can do really slick NoSQL style stuff with Postgres HStore and JSONB columns. There are a lot more data-store-as-a-service offerings for NoSQL but I don't find any of them particularly beneficial for "plain" apps but agree you can get started a lot faster. Firestore is a lot of fun when you need thi

Realtime is definitely our strongest feature so right now. We started Supabase to solve a problem at our previous startup, where we built a chat application using Firebase. Within a few months we discovered our customers were receiving their chats out-of-order and it took another few more days to figure out it was due to some Firebase quirks. We migrated to Postgres and started with Triggers/NOTIFY. But that also has some limitations (8000 byte payload limit), so we implemented the Elixir server: https://github.com/supabase/realtime

We're trying to support postgREST - at the moment that means client libraries: https://github.com/supabase/postgrest-js https://github.com/supabase/postgrest-rs https://github.com/supabase/postgrest-py (coming soon) But for additional features, it's a lot of UX - auto-docs, API gateways, auto-UI (like Airtable). For distinct features, check out our realtime server: https://github.com/supabase/realtime

Elixir really is the perfect tool for this job. Also there are a few features we are building into the realtime server that will make the system really shine (and extensible): https://github.com/supabase/realtime/issues/33 Basically we are refactoring it so you can pipe your database changes anywhere - webhooks, kafka, serverless, slack etc

This is actually how we started. I was using Firebase and hit some scale problems. We decided to migrate to Postgres, and I still needed the realtime functionality. I started naively: just with triggers/NOTIFY. There were a few tedious things about this approach (creating a trigger for every table), and one major flaw: NOTIFY has an 8000 byte payload limit. So we were getting dropped notifications. I looked into a few other ways to do it, but nothing fit 100%. So I built it, but of course with the help of other awesome opensource libraries. Using the logical decoding is an amazing solution, mostly because you get message replay if there is an outage. More details in the repo: https://github.com/supabase/realtime

Actually we chose Postgres specifically because it is scalable. I had to switch from Firebase in my previous company because we hit some scaling limits. Postgres handles the scale no problem - it's crazy the performance improvement we saw (and it's ACID) The trickiest part was the realtime functionality - https://github.com/supabase/realtime

This is very cool OP. Amazing work. For anyone looking for something similar, we are adding streams to Postgres - https://github.com/supabase/realtime It isn’t an exact comparison but it has some of the features you see here too - you can use wildcards listen to: all changes in your database, schema-level, table-level changes, or row-level. The server is built with Elixir, so you can have thousands of listeners connected via web sockets. It’s not as lightweight as OP’s solution but PG is very familiar.

I’m working on making Postgres a real-time database (like Firebase): https://github.com/supabase/realtime At its core it’s an elixir server which listens to Postgres’s built-in replication functionality, converts the byte stream to JSON, then blasts it out over websockets. It’s working in production at a few companies now. My next goal is to build connectors to other systems like Kafka, SQS etc

For anyone that might be considering trying something similar with their own Postgres database (PG10+), we recently opensourced this: https://github.com/supabase/realtime It's an Elixir (Phoenix) server that listens to PostgreSQL's native replication, transforms it into JSON, then blasts it over websockets. I see that Materialize are using Debezium, which will give you a similar result, just with connectors to Kafka etc