parseable

For write once, read many, immutable data types, Parquet on S3 is one of the most economical and scale approach. Every organisation I have talked to, in the last few years has attempted some form of Parquet on S3 implementation. However, these initiatives face challenges in standardising the approach for both ingesting and querying the data. Data quality, data governance and access control - everything becomes a challenge. We at Parseable are taking the Parquet on S3 approach and making it into an Enterprise grade product. Integrate easily with existing log agents and libraries. Built-in RBAC and user management. OIDC integration. Seamless querying by SQL. Built-in console to query and analyse this data. And much more. Please take a look!

How does this compare to Parseable? https://github.com/parseablehq/parseable First guess is that the underlying storage / query layer is pretty similar (Parquet + Datafusion), but OpenObserve has more built in use cases? As an aside, it’s awesome that Datafusion’s existence and maturity makes launching a product with scalable analytical reads 10x easier than before and cool to see so many projects integrating it

Parseable is an open source Loki alternative. - Single binary - Written in Rust (lightweight, fast and stable) - Use S3 bucket or Mount point - Visualize with Grafana https://github.com/parseablehq/parseable (founder here)

Founder here, Parseable is a lightweight log ingestion and query engine written in Rust. Parseable can ingest data from existing logging agents (FluentBit, LogStash, Vector, syslog-ng and more) using HTTP + JSON output. Ingested logs are stored as semi-indexed Parquet files (on disk or S3). You can query the data with builtin query engine using SQL or use a query engines of choice like Spark, Presto, Trino and so on. We also developed a Grafana data source plugin that lets you visualise log data via Grafana. Sample dashboard link in readme. As log data grew, our industry has responded with SaaS and fully managed offerings, but if you're looking for freedom, interoperability and full control over your data - there are no great options. With Parseable, we are looking to provide an strong, viable and FOSS alternative.