HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
eulerto
wal2json
C2019–2024
eulerto / wal2json

wal2json

JSON output plugin for changeset extraction

5Distinct people
5Mentions
0Explicit recommenders
1.5KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

JSON output plugin for changeset extraction

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 1:49:46 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year5 dated mentions

Observed mentions: 2019-02-14 22:31:34 UTC to 2024-11-04 15:32:11 UTC.

0122019-01-01 00:00:00 UTC to 2020-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments20192020-01-01 00:00:00 UTC to 2021-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2021-01-01 00:00:00 UTC to 2022-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments20212022-01-01 00:00:00 UTC to 2023-01-01 00:00:00 UTC (exclusive): 2 mentions; 1 submissions, 1 comments2023-01-01 00:00:00 UTC to 2024-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-01-01 00:00:00 UTC to 2025-01-01 00:00:00 UTC (exclusive): 2 mentions; 0 submissions, 2 comments2024

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
20192019-01-01 00:00:00 UTC2020-01-01 00:00:00 UTC101
20202020-01-01 00:00:00 UTC2021-01-01 00:00:00 UTC000
20212021-01-01 00:00:00 UTC2022-01-01 00:00:00 UTC000
20222022-01-01 00:00:00 UTC2023-01-01 00:00:00 UTC211
20232023-01-01 00:00:00 UTC2024-01-01 00:00:00 UTC000
20242024-01-01 00:00:00 UTC2025-01-01 00:00:00 UTC202

What Hacker News says

5 sources
That's a great summary! You're right, at scale, Kafka allows you to parallelise the workload more efficiently, as well as supporting multiple consumers for your replication slot. However, you should still be able to use the plug and play output processors with Kafka just as easily (implementation internally is abstracted, it works the same way with/without Kafka). We currently use the search indexer with Kafka at Xata for example. As long as the webhook server has support for rate limiting, it should be possible to handle a large amount of event notifications. Regarding the transaction metadata, the current implementation uses the wal2json postgres output plugin (https://github.com/eulerto/wal2json), and we don't include the transactional metadata. However, this is something that would be easy to enable, and integrate into the pgstream p
Read the rest
ipeline if needed, to ensure transactional consistency on the consumer side. Thanks for your feedback!
In Postgres Webhooks with Pgstream
wenbo·Apr 11, 2022Submission

Show HN: Multiplayer Demo Built with Elixir

Hey HN, I’m an engineer at Supabase [0] and one of the creators of this demo. My team and I have been working hard to bring developers the next version of Supabase Realtime. The current version of Realtime [1] is a Change Data Capture (CDC) server for a PostgreSQL database that broadcasts changes via WebSockets to authorized subscribers. It’s written in Elixir/Phoenix. The server utilizes PostgreSQL’s logical replication functionality, which writes database changes to Write-Ahead Logging (WAL) segment files, and a replication slot, responsible for managing and retaining WAL files. Database changes are polled from WAL by the server using PostgreSQL’s replication function pg_logical_slot_get_changes and changes converted to JSON objects using the wal2json [2] extension by setting it as the output plugin. Security is enforced through two che
Read the rest
cks - each check ensures only authorized client subscribers are sent database changes. The first check validates a JWT that is sent by clients subscribing to database changes. This JWT must contain an existing database role and optional claims, both of which can be referenced in Row Level Security (RLS) policies. Every valid client subscription is then inserted into the realtime.subscription table with an assigned UUID, database role, and claims. The second check calls the realtime.apply_rls SQL function from Write Ahead Log Realtime Unified Security (WALRUS) utility lib [3]. This function takes the database changes, executes a prepared statement to verify if the database role and claims have SELECT permissions on the changes, and outputs an array of authorized UUIDs. Then, the server finds all the subscribers whose UUIDs are in that array and broadcasts the changes to them. The next version of Supabase Realtime will offer three features: Broadcast, Presence, and Extensions. Broadcast, our Pub/Sub offering, can be used to pass ephemeral data from client to client such as cursor movements. This runs on a distributed cluster of nodes built on top of Phoenix PubSub + Channels. Presence, can be used for tracking online/offline users and their state. This is built into Phoenix, and uses replicated state across a cluster using an Observe-Remove-Set-Without-Tombstones (ORSWOT) CRDT [4] which prefers adds over removes when resolving conflicts. Extensions, are a way for the community to add additional functionality to take advantage of the WebSocket infrastructure. We have converted the existing Change Data Capture system to an extension that supports connecting to multiple customer databases (multi-tenancy). Other possible extensions include listening to other databases like MySQL and getting stock market events server-side [5], then broadcasting them to connected clients. This demo is built using a Supabase project, Supabase Realtime, and Next.js and deployed on 20 Fly [6] nodes located around the world. You can find an introduction and walkthrough of the demo here [5]. Supabase Realtime is entirely open source and you can find the demo code here [7]. Once we have stabilized the release we will add it to the self-hosted offering [8]. This demo is a way to highlight the upcoming features and gather feedback/ideas. Feel free to ask me anything and let me know what you think! [0] https://supabase.com [1] https://github.com/supabase/realtime [2] https://github.com/eulerto/wal2json [3] https://github.com/supabase/walrus [4] https://gsd.di.uminho.pt/members/cbm/ps/delta-crdt-draft16ma... [5] https://supabase.com/blog/2022/04/01/supabase-realtime-with-... [6] https://fly.io [7] https://github.com/supabase/realtime/tree/multiplayer [8] https://github.com/supabase/supabase/tree/master/docker
221 points · Open discussion
This is super helpful. Agreed with other comments about making this available more readily on the DO Managed DB pages. Having looked through the set of extensions, wal2json (https://github.com/eulerto/wal2json). That makes sense since there's no way at the moment to get access to replication. However, the use case for wal2json in addition to the access to replication is that it's the best way that I know of to get reliable change notifications from the database. I personally use this to pipe the JSON blobs into message queues to be consumed by applications. As a short aside, I don't use PostgreSQL's NOTIFY/LISTEN because, as I understand it, you will not receive messages if there are no connections to the DB -- this is a show stopper for me. AWS RDS allows this and Google's PostgreSQL does not. I've personally abandoned Google's version f
Read the rest
irst partly because it seems to have stalled in terms of upgrades, and AWS for being way too expensive for what you get. Although it doesn't work for me at the moment, I'm hoping the DO option will work for me in this regard in the not so distant future.
In Fully managed PostgreSQL databases

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.