HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
onyx-platform
onyx
Clojure2014–2018
onyx-platform / onyx · Archived

onyx

Distributed, masterless, high performance, fault tolerant data processing

6Distinct people
11Mentions
0Explicit recommenders
2KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Distributed, masterless, high performance, fault tolerant data processing

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 3:56:07 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year11 dated mentions

Observed mentions: 2014-10-09 15:58:44 UTC to 2018-09-12 11:06:08 UTC.

02462014-01-01 00:00:00 UTC to 2015-01-01 00:00:00 UTC (exclusive): 1 mentions; 1 submissions, 0 comments20142015-01-01 00:00:00 UTC to 2016-01-01 00:00:00 UTC (exclusive): 7 mentions; 3 submissions, 4 comments2016-01-01 00:00:00 UTC to 2017-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments20162017-01-01 00:00:00 UTC to 2018-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2018-01-01 00:00:00 UTC to 2019-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2018

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
20142014-01-01 00:00:00 UTC2015-01-01 00:00:00 UTC110
20152015-01-01 00:00:00 UTC2016-01-01 00:00:00 UTC734
20162016-01-01 00:00:00 UTC2017-01-01 00:00:00 UTC101
20172017-01-01 00:00:00 UTC2018-01-01 00:00:00 UTC101
20182018-01-01 00:00:00 UTC2019-01-01 00:00:00 UTC101

What Hacker News says

11 sources
Low signal-to-noise posts about Clojure are damaging to adoption because it doesn't say anything about why Clojure is a good option on the JVM. Instead, I will try to give some reasons why Clojure is a good fit for high-performance stream fusion, which I presume AppsFlyer has to do a lot of: - [Onyx](https://github.com/onyx-platform/onyx) for distributed, masterless, fault-tolerant data processing - [core.async](https://github.com/clojure/core.async) for CSP-style events through a channel abstraction. I miss core.async so much in every other language, I generally end up implementing a subset of the semantics. - [Transducers](https://clojure.org/reference/transducers) for composable transformations, e.g. (filter odd?) ;; returns a transducer that filters odd (map inc) ;; returns a mapping transducer for incrementing (take 5
Read the rest
) ;; returns a transducer that will take the first 5 values of a sequence (def xf (comp (filter odd?) (map inc) (take 5))) (into [] xf (range 100)) ;; => [2 4 6 8 10] Note that no intermediate data structures are created during these transformations, so it's super fast. - [Specter](https://github.com/nathanmarz/specter) decomplects navigation and transformation, pulling apart the "where" from the "how". In most imperative languages whenever the shape or performance characteristics your data structures change, you also have to update all the code that transforms it. Not with Specter. - Zach Tellman's [Manifold](https://github.com/ztellman/manifold) provides event-streaming abstractions on top of async workflows like lazy sequences, core.async and Java's BlockingQueues.
In Why Clojure?
EC2 is a proprietary service for creating virtual machines on demand in Amazon's infrastructure, it is not a computation system. Much as a relational database system provides a framework for storing and retrieving relational data, a computation system provides a framework for performing (usually large-scale) computations, like transforming text files. It is open-ended, just as an RDBMS imposes no schema. A simple example would the processing pipeline of some web crawler. Hadoop is one of the better known computation systems, though it does other things too (e.g. persistence with HDFS). Apache Storm is another popular computation system, particularly in the Clojure world. Onyx was created a few years ago, is implemented in Clojure, and competes with Storm (it's the first such system mentioned in the "What is it?" section of https://github.
Read the rest
com/onyx-platform/onyx). Onyx is distributed (meaning there are multiple nodes cooperating, typically over a network), and Jepsen is a rigorous tool for testing fault-tolerance in such systems. Being masterless (no node has central authority) is valuable for fault-tolerance. It means that in the event of a network partition, either side of the partition can continue processing as normal and recover when the partition is resolved. Jepsen simulates these and other fault conditions.
In Onyx Straps in For a Jepsening
I'll paraphrase a few snippets from my own documentation to answer these questions. Happy to comment more if needed. Information models are often superior to APIs, and almost always better than DSLs. The hyper-flexibility of a data structure literal allows Onyx workflows and catalogs to be constructed at a distance, meaning on another machine, in a different language, by another program, etc. Contrast this to Storm. Topologies are written with functions, macros, and objects. These things are specific to a programming language, and make it hard to work at a distance - specifically in the browser. JavaScript is the ultimate place to be when creating specifications. Further, the information model for an Onyx workflow has the distinct advantage that it's possible to compile other workflows (perhaps a datalog) into the workflow that Onyx under
Read the rest
stands. See https://github.com/MichaelDrogalis/onyx/blob/0.5.x/doc/user-... for a continued explanation of why Onyx is more of an "open" concept.
In Onyx: fault tolerant data processing for Clojure

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.