redpanda

Your dream has arrived! Redpanda [1] is a non-Java, ZooKeeper-free streaming platform. It's Kafka compatible, so you can use any Kafka client, of which there are many. Finally, and perhaps most importantly, it is substantially faster than Kafka. Disclosure: I work there. --- [1] https://github.com/vectorizedio/redpanda

for the record, you can do compaction with redpanda - all the same compaction strategies you know w/ kafka work w/ redpanda. https://github.com/vectorizedio/redpanda/tree/dev/src/v/storage - see all 'compaction_*' for deets

I'm really impressed with Seastar [1], the library ScyllaDB wrote to solve for writing non-blocking, asynchronous code in C++. It's also used by Redpanda [2], a Kafka-compatible service in the same vein as MemoryDB/Skyhook. [1] https://github.com/scylladb/seastar [2] https://github.com/vectorizedio/redpanda

In most cases, a single process app sending data to a file and a small HTTP server serving it will perform much better with less downtime. As I understand it, this is where Redpanda enters the picture. Maybe others here disagree? https://softwareengineeringdaily.com/2021/01/22/redpanda-kaf... https://github.com/vectorizedio/redpanda https://vectorized.io/

"Amazon Announces Grafana Fork , Promises Amazing Things for the Community" (2021/05/08) [1] On a more serious note: there is a bit of a trend in the licensing world. Mongo or ElasticSearch changes, or cool new projects like redpanda [2] launching with BSL. Open source is still a very young business model. One that has provided a tremendous amount of value over the past decades, but we still need to figure out how to build sustainable businesses with it. Be it open core, more restrictive licenses, copyleft, dual licensing, ... I just really hope that we don't slide back into the world of proprietary closed source. The danger is certainly there, and it will hit smaller companies the most. [1] https://aws.amazon.com/grafana/ [2] https://github.com/vectorizedio/redpanda

The BSL license makes choosing it "a discussion" versus using OSI licensed software: https://github.com/vectorizedio/redpanda/tree/v21.3.7/licens... I don't have enough experience with these new vanity licenses to know what the contribution story looks like, either

that's correct - we actually don't manually write the kafka api, but we codegenerate it: https://github.com/vectorizedio/redpanda/blob/dev/src/v/kafk... the tl;dr is that we wrote a pretty fast fragmented buffer parser (due to our thread-per-core design), we consume some json files and codegen it.... which basically means, if you connect an existing app that uses the kafka driver and it doesn't work it's a bug, and it's a matter of teaching the codegen to fix it most of the time. Transactions should land in the next month or so, which is the only thing really missing today. we have a ton of solo devs typically that come from the Go, Rust, JS & Python seem to like it quite a bit because of what you said.