HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
asavinov
bistro
Java2017–2019
asavinov / bistro

bistro

A general-purpose data analysis engine radically changing the way batch and stream data is processed

Data & DatabasesRobotics & IoT
Explore on GitHubRead Hacker News discussions
2Distinct people
15Mentions
0Explicit recommenders
8GitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

A general-purpose data analysis engine radically changing the way batch and stream data is processed

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 3:56:07 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by month15 dated mentions

Observed mentions: 2017-12-31 18:05:32 UTC to 2019-01-30 10:33:01 UTC.

01232017-12-01 00:00:00 UTC to 2018-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 commentsDec 20172018-01-01 00:00:00 UTC to 2018-02-01 00:00:00 UTC (exclusive): 2 mentions; 1 submissions, 1 comments2018-02-01 00:00:00 UTC to 2018-03-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2018-03-01 00:00:00 UTC to 2018-04-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2018-04-01 00:00:00 UTC to 2018-05-01 00:00:00 UTC (exclusive): 2 mentions; 0 submissions, 2 comments2018-05-01 00:00:00 UTC to 2018-06-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2018-06-01 00:00:00 UTC to 2018-07-01 00:00:00 UTC (exclusive): 3 mentions; 1 submissions, 2 commentsJun 20182018-07-01 00:00:00 UTC to 2018-08-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2018-08-01 00:00:00 UTC to 2018-09-01 00:00:00 UTC (exclusive): 2 mentions; 0 submissions, 2 comments2018-09-01 00:00:00 UTC to 2018-10-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2018-10-01 00:00:00 UTC to 2018-11-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2018-11-01 00:00:00 UTC to 2018-12-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2018-12-01 00:00:00 UTC to 2019-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2019-01-01 00:00:00 UTC to 2019-02-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 commentsJan 2019

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
Dec 20172017-12-01 00:00:00 UTC2018-01-01 00:00:00 UTC101
Jan 20182018-01-01 00:00:00 UTC2018-02-01 00:00:00 UTC211
Feb 20182018-02-01 00:00:00 UTC2018-03-01 00:00:00 UTC000
Mar 20182018-03-01 00:00:00 UTC2018-04-01 00:00:00 UTC000
Apr 20182018-04-01 00:00:00 UTC2018-05-01 00:00:00 UTC202
May 20182018-05-01 00:00:00 UTC2018-06-01 00:00:00 UTC101
Jun 20182018-06-01 00:00:00 UTC2018-07-01 00:00:00 UTC312
Jul 20182018-07-01 00:00:00 UTC2018-08-01 00:00:00 UTC101
Aug 20182018-08-01 00:00:00 UTC2018-09-01 00:00:00 UTC202
Sep 20182018-09-01 00:00:00 UTC2018-10-01 00:00:00 UTC101
Oct 20182018-10-01 00:00:00 UTC2018-11-01 00:00:00 UTC000
Nov 20182018-11-01 00:00:00 UTC2018-12-01 00:00:00 UTC000
Dec 20182018-12-01 00:00:00 UTC2019-01-01 00:00:00 UTC101
Jan 20192019-01-01 00:00:00 UTC2019-02-01 00:00:00 UTC101

What Hacker News says

15 sources
asavinov·Dec 3, 2018
Lambdo is a workflow engine which simplifies data analysis by combining in one analysis pipeline * Feature engineering and machine learning: Lambdo does not distinguish them and treats them as data transformations * Model training and prediction: both feature definitions and ML models can be trained as part of one workflow * Table population and column evaluation: workflow consists of nodes of these two types. This makes it similar to Bistro: https://github.com/asavinov/bistro Lambdo is intended for the following use cases: * Numerous derived features with parameters derived from the data * Regular re-training is required by using the same features as those to be used during prediction * Time series analysis because it is where the quality of derived features is especially important * Customization via user-defined Python functions
In Show HN: Lambdo – Feature engineering and machine learning together
asavinov·Aug 15, 2018
There are two general (potential) problems due to the use of (multiple) joins: * run time: performance disaster * design time: conceptual chaos Some of them are analyzed in [1] where it is essentially argued that join considered harmful and a join-free (column-oriented) approach is described. This approach has been implemented in [2] which is a library for batch and stream data processing, and in [3] which is a library for data analysis. Their main common unique feature is that they rely on column operations, that is, data processing is described as a graph of operations with columns as opposed to using set operations like join. [1] https://www.researchgate.net/publication/301764816_Joins_vs_... Joins vs. Links or Relational Join Considered Harmful [2] https://github.com/asavinov/bistro - A general-purpose data analysis engine radically c
Read the rest
hanging the way batch and stream data is processed [3] https://github.com/asavinov/lambdo - A column-oriented approach to feature engineering. Feature engineering and machine learning: together at last! (Disclaimer: I am an author)
In Cost of a Join
asavinov·Jun 13, 2018
Bistro Streams [1] is similar to Kafka Streams in how it is supposed to be used - it is implemented as a library which can be part of an application (including IoT analytics) or run at the edge, for example, on gateways or devices. However, Bistro Streams is based on different principles and has the following major distinguishing features: o [Column-oriented logical model] Bistro Streams describes its data processing logic using mainly column operations as opposed to set operations [2]. It is a unique feature of this system: no joins, no group-by, no map-reduce. o [Column-oriented physical model] Data within the system is represented in columns (in-memory). It is apparently not new and widely used in column stores but it is new for stream processing. In the case of long histories (which is needed for complex analysis) and complex analytic
Read the rest
workflows it can provide higher performance. It is also important for running on edge devices with limited resources. o [Separate injection, processing, retention] Bistro Streams separates the logic of (1) triggering the processes for appending, evaluating, and deleting data from the logic of (2) what to do during evaluation (data processing itself). In particular, the frequency and conditions for starting evaluations are specified using a separate API. The same for retention policy where deletion time is determined not by the windows used during its processing (say, for moving average) but separately. I am the author of Bistro Streams [1] and the underlying Bistro Engine [2]. I will be glad to answer questions and any feedback is welcome. In particular, what are possible application areas for this system. [1] Bistro Streams: https://github.com/asavinov/bistro/tree/master/server [2] Bistro Engine: https://github.com/asavinov/bistro/blob/master/core
In Show HN: Bistro Streams – A light-weight column-oriented stream analytics server
asavinov·Jun 12, 2018
The question whether we need distributed stream processing can hardly be answered unambiguously. In general, everything depends on what features we actually need: scalability, fault tolerance, high availability, performance (latency or throughput) etc. In any case, there are two ways to distribute stream processing tasks: * [~Heavyweight] Use a cluster with multiple runners like [Spark Streaming](https://spark.apache.org/streaming/) or [Flink](https://flink.apache.org/) * [~Lightweight] Do stream processing in applications, at the edge, on gateways or devices like [Bistro Streams](https://github.com/asavinov/bistro/tree/master/server) or [Kafka Streams](https://kafka.apache.org/documentation/streams/) Normally, distributed stream processing requires also partitioning the data, e.g., by user or session ids, so that these isolated streams c
Read the rest
an be processed independently at different nodes.
In Do We Need Distributed Stream Processing?
asavinov·May 12, 2018
Sad to see it go. I had professional interest to Fieldbook because I had been developing a similar "spreadsheet"-like application intended for radically changing the way table table is processed - http://conceptoriented.com - and failed. Finally, I decided to move this (quite unique) technology to completely different area and re-implement it as a kind of major alternative to MapReduce focused on general-purpose batch and stream processing (mainly IoT and edge analytics): https://github.com/asavinov/bistro I am curious to see what happens with other apps like airtable, rowshare or zenkit and the whole market. Will they be able to survive as independent companies (like Tableau or Qlik in self-service BI)?
In Fieldbook is shutting down
asavinov·Apr 18, 2018
> Giving meaning to ... Ingesting such amounts of data is a challenge indeed. But problems will become much more complicated if it is necessary to perform complex analysis during data ingestion. Such analysis (not simply event pre-processing) can arise because of the following reasons: * It is physically not possible to store this amount of events. For example, assume you collect them from devices and sensors * It is necessary to make faster decisions, e.g., in mission critical applications * It can be more efficient to do some analytics before storing data (as opposed to first storing data persistently and then loading it again for analysis) Such analysis can be done by conventional tools like Spark Streaming (micro batch processing) or Kafka Streams (works only with Kafka). One novel approach is implemented in Bistro Streams [0] (I am a
Read the rest
n author). It is intended for general-purpose data processing including both batch and stream analytics but it radically differs from MapReduce, SQL and other set-oriented data processing frameworks. It represents data via functions and processes data via column operations rather than having only set operations. [0] Bistro: https://github.com/asavinov/bistro
In Giving meaning to 100B analytics events a day with Kafka, Dataflow and BigQuery
asavinov·Dec 31, 2017
I finished Data Commander (MVP) - http://conceptoriented.com - A web-app for column-oriented data transformations. After getting some feedback I decided to position this technology differently. Instead of exposing the functionality via web app, I started implementing a Java library http://github.com/asavinov/bistro - an alternative to map-reduce. In 2018 I am going to develop a server for IoT and stream analytics - an alternative to kafka stream analytics based on Bistro.
In Ask HN: What did you work on in 2017?

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.