quix-streams

Congratulations on launching your project! We spoke back in March at a Kafka Summit London social meetup and talked all things Python and Kafka (I work on https://github.com/quixio/quix-streams). Always great to see a new stream processing project tackle a new segment

Hey all! We’ve built a Pandas-like interface to make it easy to work with streaming data using what we call ‘Streaming DataFrames’. For example, suppose that you want to convert speed measurement units from meters per second to kilometers per hour With static data in Pandas, you’d do this: df["speed_km_h"] = df["speed_m_s"] * 3.6 With Streaming DataFrames, it’s pretty much the same thing… sdf["speed_km_h"] = sdf["speed_m_s"] * 3.6 …except it’s being done continuously and the updated records can be sent to an output topic in Kafka with almost no delay after they’ve been processed. You can also do a stateful tumbling window like this: sdf = sdf.apply(lambda row: row["speed_km_h"]).tumbling_window(timedelta(seconds=3O), grace_ms=timedelta(seconds=1).mean().final() I know some peeps might be thinking “eeeeww Pandas”, but we wanted to make it

Hello HN, we’re open sourcing a new library called Quix Streams (https://github.com/quixio/quix-streams)—a high-performance Python and C# library that processes time-series telemetry data with message broker such as Apache Kafka, instead of a database. We’re a team of engineers who all worked together at the Formula 1 McLaren pit wall. We struggled to get wanted we wanted out of existing stream processing tech, so we built our own platform called Quix. Although our original requirements were pretty niche (process over 60k data points per second), our solution is designed to make life easier for anyone who needs to process high volumes of time-series data. We also wanted it to be as accessible as possible for data scientists and ML engineers (e.g. those who run a lot of physics models on sensor data). This means it’s got a lot of built-in