HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
pystorm
streamparse
Python2014–2021
pystorm / streamparse

streamparse

Run Python in Apache Storm topologies. Pythonic API, CLI tooling, and a topology DSL.

2Distinct people
8Mentions
0Explicit recommenders
1.5KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Run Python in Apache Storm topologies. Pythonic API, CLI tooling, and a topology DSL.

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

Mentioned in the same Hacker News threads. That is not proof they are substitutes.

This week on Hacker News

Nov 9–Nov 16 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/16/2026, 3:16:21 AM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year8 dated mentions

Observed mentions: 2014-05-11 15:10:56 UTC to 2021-07-13 14:29:01 UTC.

012342014-01-01 00:00:00 UTC to 2015-01-01 00:00:00 UTC (exclusive): 2 mentions; 1 submissions, 1 comments20142015-01-01 00:00:00 UTC to 2016-01-01 00:00:00 UTC (exclusive): 4 mentions; 0 submissions, 4 comments2016-01-01 00:00:00 UTC to 2017-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2017-01-01 00:00:00 UTC to 2018-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments20172018-01-01 00:00:00 UTC to 2019-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2019-01-01 00:00:00 UTC to 2020-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2020-01-01 00:00:00 UTC to 2021-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2021-01-01 00:00:00 UTC to 2022-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2021

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
20142014-01-01 00:00:00 UTC2015-01-01 00:00:00 UTC211
20152015-01-01 00:00:00 UTC2016-01-01 00:00:00 UTC404
20162016-01-01 00:00:00 UTC2017-01-01 00:00:00 UTC101
20172017-01-01 00:00:00 UTC2018-01-01 00:00:00 UTC000
20182018-01-01 00:00:00 UTC2019-01-01 00:00:00 UTC000
20192019-01-01 00:00:00 UTC2020-01-01 00:00:00 UTC000
20202020-01-01 00:00:00 UTC2021-01-01 00:00:00 UTC000
20212021-01-01 00:00:00 UTC2022-01-01 00:00:00 UTC101

What Hacker News says

1 sources
Wonder why this is getting posted today in particular? The quick summary here is that this was a clean-house rewrite of Apache Storm done by an internal team at Twitter. As an open source project history refresher, Apache Storm was originally built by a startup called Backtype, and the project was led by Nathan Marz, the technical founder of Backtype. Then, Backtype was acquired by Twitter, and Storm became a major component for large-scale stream processing (of tweets, tweet analytics, and other things) at Twitter. I wrote a summary of the "interesting bits" of Apache Storm here: https://blog.parse.ly/storm/ However, at a certain point, Nathan Marz left Twitter, and a different group of engineers tried to rethink Storm inside Twitter. There was also a lot of work going on around Apache Mesos at the time. Heron is kind of a merger of thei
Read the rest
r "rethinking" of Storm while also making it possible to manage Storm-like Heron clusters using Mesos. But, I don't think Heron really took off. Meanwhile, Storm got very, very stable in the 1.x series, and then had a clean-house rewrite from Clojure to Java in the 2.x series, mainly to improve performance even more. The last stable/major Storm release was in 2020. Storm provides a stream processing programming API, a multi-lang wire protocol, and a cluster management approach. But certain cluster computing problems can probably be better solved at the infrastructure layer today. (For example, Storm was developed before the whole container + docker + k8s focus in cloud ops.) That said, it's still a very powerful system; on my team, we process 75K+ events per second across hundreds of vCPU cores and thousands of Python processes with sub-second latencies by combining Storm and Kafka with our open source Python project, streamparse. https://github.com/Parsely/streamparse The core problems Storm solves: modeling data processing as a computation graph; high-speed network communication between threads, processes, and nodes; message delivery guarantees and retry capabilities; tunable parallelism; built-in monitoring and logging; and much more. (Also, I'd be remiss if I didn't mention -- if you're interested in stream processing and distributed computing, we are hiring Python Data Engineers to work on a stack involving Storm, Spark, Kafka, Cassandra, etc.) -- https://www.parse.ly/careers/python_data_engineer
In Apache Heron: A realtime, distributed, fault-tolerant stream processing engine

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.