proton

Temporal binding is one of the hardest problems in visualization, all about aggregations across different time windows. And it gets even harder when the clock never stops and live data keeps flowing. Client visualization layer: Timeplus Vistral (https://github.com/timeplus-io/vistral) Server data processing layer: Timeplus Proton (https://github.com/timeplus-io/proton) Streaming-native from processing to insights in motion

I work on Proton/Timeplus (disclosure). We hit a SIGSEGV where the backtrace was misleading: our fatal handler tried to print a stack trace and that trace capture sometimes crashed inside libunwind, so it looked like “unwinding is broken”. What worked was building a deterministic postmortem harness (core dump + debug binary + symbols + matching source paths) inside Docker, then installing Codex in the same container so it could run GDB + rebuild/iterate in-place. OpenAI Codex pivoted away from unstable backtraces and classified the crash via siginfo_t/ucontext_t. It turned out to be SEGV_PKUERR (Intel MPK/PKU) caused by a thread-local PKRU mismatch when some worker threads entered V8. PR with the patch: https://github.com/timeplus-io/proton/pull/1091

Two years after open sourcing Proton, we're releasing v3.0 which brings enterprise-grade streaming capabilities to the open source version: full connectivity, processing, and routing in a single binary with zero dependencies. Key features: First vectorized streaming SQL engine in modern C++ with JIT compilation High-throughput, low-latency, high-cardinality processing End-to-end streaming: ETL, joins, aggregation, alerts, and tasks Native connectors: Kafka, Redpanda, Pulsar, ClickHouse, Splunk, Elastic, MongoDB, S3, Iceberg Native Python UDF/UDAF support to support your AI/ML work loads The same performance we've proven in large enterprise deployments is now available in the community edition. Would love feedback from anyone working with streaming data or looking for Flink/ksqlDB alternatives.

you can check https://github.com/timeplus-io/proton which provides streaming processing pipeline.

Dynamic Tables are interesting for declarative streaming. In the ClickHouse ecosystem, you might want to look at materialized views combined with streaming engines. For real-time transformations, there are a few approaches: - Native ClickHouse MaterializedViews with AggregatingMergeTree - Stream processors that write to ClickHouse (Flink, Spark Streaming) - Streaming SQL engines that can read/write ClickHouse We've been working on streaming SQL at Proton (github.com/timeplus-io/proton) which handles similar use cases - continuous queries that maintain state and can write results back to ClickHouse. The key difference from Dynamic Tables is handling unbounded streams vs micro-batches. What's your specific use case? Happy to discuss the tradeoffs.

Existing OSS C++ projects like ClickHouse and DuckDB support reading from Iceberg tables. Writing requires Spark, PyIceberg, or managed services. In this PR https://github.com/timeplus-io/proton/pull/928, we are open-sourcing a C++ implementation of Iceberg integration. It's an MVP, focusing on REST catalog and S3 read/write(S3 table support coming soon). You can use Timeplus to continuously read data from MSK and stream writes to S3 in the Iceberg format. No JVM. No Python. Just a low-overhead, high-throughput C++ engine. Docker/K8s are optional. Demo video: https://www.youtube.com/watch?v=2m6ehwmzOnc Help us improve the code to add more integrations and features. Happy to contribute this to the Iceberg community. Or just roast the code. We’ll buy the virtual coffee.

In 2023, AWS announced[1] IAM authentication for MSK Kafka clusters with support for "all programming languages"… except C++. While Java[2], Python[3], Go[4], and others got official SDKs, C++ developers/vendors were stuck hardcoding SCRAM-SHA credentials in code/configs or relying on heavier Java-based tools like Kafka Connect or Apache Flink. Later, community projects added Rust[5] and Ruby[6] support. Why no C++? Rust might be the new favorite, but C++ is still king for high-performance data systems: minimal dependencies, lean resource use, and raw speed. At Timeplus, we needed IAM auth for our C++ streaming engine, Proton, so we built it ourselves. Today, we’re open-sourcing our code for AWS MSK IAM authentication. It’s live in Timeplus Proton 1.6.12 Just attach an IAM role to your EC2 instance or EKS pod, then put the Timeplus Proton

Timeplus Proton [1] is an OSS fork of Clickhouse that adds support for streaming queries. Timeplus Proton is wire-compatible with Clickhouse and its streaming support makes the log tailing use case you mentioned above easy to setup: > - You can live tail events, I don't think Grafana for Clickhouse has that (I'm a big fan of tail -f) So it sounds like your v2 will work with any DB that is wire-compatible with Clickhouse, correct? [1] https://github.com/timeplus-io/proton

Timeplus proton OSS https://github.com/timeplus-io/proton does similar thing but with powerful historical query processing as well.

What's the process for adding support for other databases to your tool qStudio? I'm thinking perhaps you could add support for Timeplus [1]? Timeplus is a streaming-first database built on ClickHouse. The core DB engine Timeplus Proton is open source [2]. It seems that qStudio is open source [3] and written in Java and will need a JDBC driver to add support for a new RDBMS? If yes, Timeplus Proton has an open source JDBC driver [4] based on ClickHouse's driver but with modifications added for streaming use cases. 1: https://www.timeplus.com/ 2: https://github.com/timeplus-io/proton 3: https://github.com/timeseries/qstudio 4: https://github.com/timeplus-io/proton-java-driver

A deep dive from CTO of Timeplus. ksqlDB is great for * SQL Interface * Real-Time Streaming Processing * Integration with Kafka * Stateful Processing * Scalability * Security However, there are also limitations with ksqlDB: * Deep Coupling with Kafka * Heavy Resource Consumption * Not Designed for Analytics * Not True Open-Source There are lots of similarities between Proton and ksqlDB. Both support: * Stateful streaming processing * Data persistence * Stream/table concept * Dual query mode, long running, unbounded push based query and bounded, pull base query * Query Kafka data and write analytic results back to Kafka Most features of ksqlDB can be found in Proton, while Proton offers additional benefits: * Proton demonstrates stronger performance * Proton is more flexible when consuming Kafka data * Proton is purpose designed for a

As a streaming SQL engine, Proton is a fast and lightweight alternative to Apache Flink, powered by ClickHouse. It can help developers solve the data streaming challenges from processing, routing to analytics, and send aggregated data to the downstream systems. Proton is also the core engine of Timeplus (http://timeplus.com), which is a cloud-native streaming analytics platform. If you are looking for a high-performance and lightweight streaming processing engine capable of running everywhere, or gaining streaming insights with historical context, you should try Proton: https://github.com/timeplus-io/proton!