denormalized

tl;dr we’re releasing python bindings for writing stream processing jobs using the Denormalized Rust library. Hey HN, A few months ago we released Denormalized, our embeddable stream processing engine written in rust [0]. Today we’re excited to showcase python bindings for the library. Python is still very much the language of data and we wanted to make it easy for more engineers to write fast streaming jobs without having to learn Rust. Currently, you can use the library to write streaming aggregations, stream joins, and user defined aggregate functions (thought the API for this is quite clunk). The library is available on pypi: `pip install denormalized` and we’ve created a simple docker container that will quickly spin up an instance of kafka along with a script to write fake data to make running the examples [1] really easy: `docker r

tl;dr we built an embeddable stream processing engine in Rust using apache DataFusion, check us out at https://github.com/probably-nothing-labs/denormalized Hey HN, We’d like to showcase a very early version of our embeddable stream processing engine called Denormalized. The rise of DuckDB has abundantly made it clear that even for many workloads of Terabyte scale, a single node system outshines the distributed query engines of previous generation such as Spark, Snowflake etc in terms of both performance and cost. Now a lot of workloads DuckDB is used for were normally considered to be “big data” in the previous generation, but no more. In the context of streaming especially, this problem is more acute. A streaming system is designed to incrementally process large amounts of data over a period of time. Even on the upper end of scale, prod