HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
nikomatsakis
rayon
Rust2014–2020
nikomatsakis / rayon

rayon

Rayon: A data parallelism library for Rust

11Distinct people
11Mentions
0Explicit recommenders
62GitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Rayon: A data parallelism library for Rust

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 3:56:07 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year11 dated mentions

Observed mentions: 2014-12-23 18:04:31 UTC to 2020-06-24 13:50:28 UTC.

0123452014-01-01 00:00:00 UTC to 2015-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments20142015-01-01 00:00:00 UTC to 2016-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2016-01-01 00:00:00 UTC to 2017-01-01 00:00:00 UTC (exclusive): 5 mentions; 0 submissions, 5 comments2017-01-01 00:00:00 UTC to 2018-01-01 00:00:00 UTC (exclusive): 4 mentions; 1 submissions, 3 comments20172018-01-01 00:00:00 UTC to 2019-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2019-01-01 00:00:00 UTC to 2020-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2020-01-01 00:00:00 UTC to 2021-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2020

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
20142014-01-01 00:00:00 UTC2015-01-01 00:00:00 UTC101
20152015-01-01 00:00:00 UTC2016-01-01 00:00:00 UTC000
20162016-01-01 00:00:00 UTC2017-01-01 00:00:00 UTC505
20172017-01-01 00:00:00 UTC2018-01-01 00:00:00 UTC413
20182018-01-01 00:00:00 UTC2019-01-01 00:00:00 UTC000
20192019-01-01 00:00:00 UTC2020-01-01 00:00:00 UTC000
20202020-01-01 00:00:00 UTC2021-01-01 00:00:00 UTC101

What Hacker News says

11 sources
Clearly, we are not the only ones feeling this way. I suspect that if PyTorch or TensorFlow were to provide support for rust as a first-class citizen they would get a whole new community of users very quickly -- people looking to write production AI/DL/ML code in a fast, practical, memory-safe, widely used language. The only other "fast, practical, memory-safe, widely used language" that looks like a real alternative to Rust is Swift. But unlike Swift, rust was designed from the ground up for for data parallelism and other forms of concurrency with "zero-cost abstractions" the allow "fearless concurrency" (e.g., take a look at https://github.com/nikomatsakis/rayon/ to get an idea of what's possible), and for predictable performance (e.g., compiled code doesn't suddenly slow-down to collect garbage or synchronize reference counts among CPU
Read the rest
s). Concurrency appears to have been an afterthought in Swift's case, explicitly: https://gist.github.com/lattner/31ed37682ef1576b16bca1432ea9... . See also https://doc.rust-lang.org/book/ch16-00-concurrency.html and https://blog.spencerkohan.com/impressions-of-rust-as-a-swift... Data parallelism and other forms of concurrency should become more and more important over time as models grow larger and larger, well beyond GPT-3's 175 billion parameters. This would favor rust.
In NPhysics: 2D and 3D Real-Time Physics Engine for Rust
I think it's fair to say that pdqsort (pattern-defeating quicksort) is overall the best unstable sort and timsort is overall the best stable sort in 2017, at least if you're implementing one for a standard library. The standard sort algorithm in Rust is timsort[1] (slice::sort), but soon we'll have pdqsort as well[2] (slice::sort_unstable), which shows great benchmark numbers.[3] Actually, I should mention that both implementations are not 100% equivalent to what is typically considered as timsort and pdqsort, but they're pretty close. It is notable that Rust is the first programming language to adopt pdqsort, and I believe its adoption will only grow in the future. Here's a fun fact: Typical quicksorts (and introsorts) in standard libraries spend most of the time doing literally nothing - just waiting for the next instruction because of
Read the rest
failed branch prediction! If you manage to eliminate branch misprediction, you can easily make sorting twice as fast! At least that is the case if you're sorting items by an integer key, or a tuple of integers, or something primitive like that (i.e. when comparison is rather cheap). Pdqsort efficiently eliminates branch mispredictions and brings some other improvements over introsort as well - for example, the complexity becomes O(nk) if the input array is of length n and consists of only k different values. Of course, worst-case complexity is always O(n log n). Finally, last week I implemented parallel sorts for Rayon (Rust's data parallelism library) based on timsort and pdqsort[4]. Check out the links for more information and benchmarks. And before you start criticizing the benchmarks, please keep in mind that they're rather simplistic, so please take them with a grain of salt. I'd be happy to elaborate further and answer any questions. :) [1] https://github.com/rust-lang/rust/pull/38192 [2] https://github.com/rust-lang/rust/issues/40585 [3] https://github.com/rust-lang/rust/pull/40601 [4] https://github.com/nikomatsakis/rayon/pull/379
In Pattern-defeating quicksort
I think some of issue here is that when you ask for "concurrency / parallel programming primitives", you're actually talking about more than one problem. We can divide things up as follows: - Data parallelism. This is where you have (for example) a giant array of data to process and want to use as many cores as possible. For this, rayon is gorgeous: https://github.com/nikomatsakis/rayon Seriously, I can just not say enough good things about programming with rayon, and how nice Rust's "no mutable aliasing" makes coding. Also at some point, Rust will also need good libraries for talking to the GPU. - I/O concurrency. This is where you have a server that needs to respond quickly to 10,000+ sockets, for example. For this, it looks like everyone is standardizing on futures and tokio: https://github.com/alexcrichton/futures-rs https://github.co
Read the rest
m/tokio-rs/tokio But it seems like you're most interested in Erlang-style actors with M:N green threads? The issue here is that—as best I remember the history of Rust—the Rust maintainers decided that green threads were simply too high a level of abstraction for Rust. I think Rust will eventually move back in that direction, first using futures+tokio+async I/O, and then by eventually adding async/await sugar on top. That way, you'll only pay for coroutines/green threads when you explicitly opt in. But it will take a year or so for all this to start maturing, I think. In the meantime, tokio's method-chaining syntax should actually be somewhat tolerable, especially because Rust has a `#[must_use]` attribute that prevents you from forgetting to use a future. If you're lazy, just run everything through rustfmt to get the indentation right.
In The Piston image library is now pure Rust
I have left more comments on this thread than I should considering my lack of experience with Rust, but I think the comparison with Scala is valid. What got me thinking about that was the implementation of into_par_iter, which is a trait implemented on a bunch of standard types, not unlike implicit methods in Scala. I know this is rare code, but reading this left with very similar feeling that I had when reading Scala standard library code: https://github.com/nikomatsakis/rayon/blob/master/src/par_it...
In 24 days of Rust – Rayon
> I don't know Scala or Erlang, but based on the results I would assume both are allocating a posix thread for each thread in the software and so it is using the kernel scheduler to switch between them with all the overhead that entails. Incorrect. > I have been considering threading in Rust vs Go and that is the biggest difference I see between the approaches. However it should be possible to implement Go style goroutines in Rust using custom dispatch code like old C coroutines. I think I have already seen libraries like that, but I haven't dug in the details to know for sure. There is https://github.com/dpc/mioco for I/O-based coroutines. For CPU bound tasks you want something like https://github.com/nikomatsakis/rayon.
In Akka, Haskell, Erlang, Go and .NET Core compared on 1M threads
> To have those guarantees the data would need to stored in an atomically reference counted structure (or something else with the same guarantees), which is just like you said. That is not true. `Send` is having its `'static` bound removed, so sharing data will no longer require reference counting for threads with bounded lifetime: https://github.com/rust-lang/rfcs/pull/458 (the RFC hasn't been officially accepted but it's been referenced as a foregone conclusion several times by Rust core team members). There is already an experimental library exploiting this for `Sync` data, which includes immutable substructures: https://github.com/nikomatsakis/rayon The only reason people aren't more aware of this is because the functionality isn't exposed in the standard library yet. The language is perfectly capable.
In Clojure: Creating a reducible repeat

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.