HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
mratsim
weave
Nim2019–2025
mratsim / weave

weave

A state-of-the-art multithreading runtime: message-passing based, fast, scalable, ultra-low overhead

9Distinct people
30Mentions
0Explicit recommenders
587GitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

A state-of-the-art multithreading runtime: message-passing based, fast, scalable, ultra-low overhead

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

Mentioned in the same Hacker News threads. That is not proof they are substitutes.

This week on Hacker News

Nov 9–Nov 16 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/16/2026, 12:01:57 AM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year30 dated mentions

Observed mentions: 2019-11-07 21:10:13 UTC to 2025-03-04 17:23:57 UTC.

051015202019-01-01 00:00:00 UTC to 2020-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments20192020-01-01 00:00:00 UTC to 2021-01-01 00:00:00 UTC (exclusive): 18 mentions; 1 submissions, 17 comments2021-01-01 00:00:00 UTC to 2022-01-01 00:00:00 UTC (exclusive): 4 mentions; 0 submissions, 4 comments2022-01-01 00:00:00 UTC to 2023-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments20222023-01-01 00:00:00 UTC to 2024-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-01-01 00:00:00 UTC to 2025-01-01 00:00:00 UTC (exclusive): 5 mentions; 0 submissions, 5 comments2025-01-01 00:00:00 UTC to 2026-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2025

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
20192019-01-01 00:00:00 UTC2020-01-01 00:00:00 UTC101
20202020-01-01 00:00:00 UTC2021-01-01 00:00:00 UTC18117
20212021-01-01 00:00:00 UTC2022-01-01 00:00:00 UTC404
20222022-01-01 00:00:00 UTC2023-01-01 00:00:00 UTC101
20232023-01-01 00:00:00 UTC2024-01-01 00:00:00 UTC000
20242024-01-01 00:00:00 UTC2025-01-01 00:00:00 UTC505
20252025-01-01 00:00:00 UTC2026-01-01 00:00:00 UTC101

What Hacker News says

18 sources
Worth pointing out that the bug is still present in latest glibcs (I can reproduce it on Fedora 33), and affects programs that use pthread_cond_signal, in particular language runtimes like OCaml, Python and .NET that have a global runtime lock. Bugreports: * Fedora: https://bugzilla.redhat.com/show_bug.cgi?id=1889892 * Ubuntu: https://bugs.launchpad.net/ubuntu/+source/glibc/+bug/1899800 Also see https://github.com/mratsim/weave/pull/58/files
In Using TLA+ in the Real World to Understand a Glibc Bug
> no data races for multithreading This is simply not true. Rust borrow checker has no notion of memory models: i.e. sequential consistency, acquire, release or relaxed semantics. The Tokio team needs an additional model checker to ensure no data race in Tokio: - https://github.com/tokio-rs/loom If you want to ensure no data races you need formal verification not a borrow checker, I've compiled my research and a RFC for Nim formal verification here: - https://github.com/mratsim/weave/issues/18 - https://github.com/nim-lang/RFCs/issues/222
In Nim 1.4
I've been diving into parallel reproducible PRNG [1] in the end I choose to have a PRNG based on xoshiro and seeding process that could take loop iteration variables [2] for reproducibility: - [1] https://github.com/mratsim/weave/issues/147 - [2] https://github.com/mratsim/trace-of-radiance/blob/99f7d85/tr... - Usage for parallel raytracing: https://github.com/mratsim/trace-of-radiance/blob/99f7d85/tr... Beyond the JAX paper, the Pedigree-based RNG from Leiserson was interesting as well: - http://supertech.csail.mit.edu/papers/dprng.pdf - https://pdfs.semanticscholar.org/a32c/3b7b8d5d3ebd594bea2d9c... And Haskell's splittable RNG: - http://www.hg.schaathun.net/research/Papers/hgs2015jfp.pdf - http://publications.lib.chalmers.se/records/fulltext/183348/... - http://gee.cs.oswego.edu/dl/papers/oopsla14.pdf
In A Family of Better Random Number Generators
I've written a multithreading runtime from the ground up in Nim.[1] And yes, writing that taught me that current programming languages are not sufficiently equipped to deal with concurrency bugs, including Rust. The bugs that I had where of the dining philosophers kind (deadlocks or livelocks when trying to put a thread to sleep for example or resolving data dependencies between multiple threads), some were due to a bug in glibc where only formal verification allowed me to ensure that I was doing the correct thing and it was the lower-level layers that was incorrect.[2] Similarly, state machine formal verification to prevent design bugs is something that hardware engineers are deeply aware of but would also be extremely useful to ensure "correct-by-construction" event driven code in software engineering. In comparison, we have many more t
Read the rest
ools to address memory bugs (Address Sanitizer, Valgrind and fuzzers in particular) but synchronization and communicating state machine bugs are still impractically addressed at the moment. This is something I'd really want Nim to explore and I'm really excited about the Z3 integration[3] to enable new formally verified use-cases[4]. Unfortunately, when I express concerns about those bugs to people from the Rust community, they tend to dismiss those as if the borrow checker was the panacea to all multithreading problems. [1]: https://github.com/mratsim/weave [2]: https://github.com/mratsim/weave/issues/56 [3]: https://nim-lang.org/docs/drnim.html [4]: https://github.com/nim-lang/RFCs/issues/222
In Rust concurrency: the archetype of a message-passing bug
And that's where model checking / formal verification is a tremendous help (or would be if it was easier to use). There are many properties of multithreaded/distributed systems that are emergent and that current static compilers don't address. In distributed systems it's becoming more common to use TLA+[1] to model the system and ensure that we don't have bugs at the design level (rather than just the system level). In particular for message-passing, I'm quite curious about the state of the research on Communicating Finite State Machines[1] and Petri Nets[2] and if there are actual products that could be integrated in regular languages and are not just of academic interest only. I expect that Ada/Sparks[1] has some support for this pattern though Ada multithreading story is not there yet (?). I'm quite excited about the future Z3 prover i
Read the rest
ntegration in Nim[5] and the potential to write concurrent programs free of design bugs[6] to address both ends of the concurrency bugs: - bugs in the low-level concurrent data structure via formal verification (which is something I used int he past to prove that my design was bug-free and that the deadlock I experienced was a condition variable bugs in Glibc[7] and I had to directly use futexes instead. - bugs in the high-level design, implemented as event-loop / finite state-machine reacting to events as received by channels Are there people working on model checkers or formal verification in Rust? Like the author, i think addressing memory bugs.the borrow checker is only a part of the "fearless concurrency" story and preventing design bugs would be extremely valuable. [1]: https://lamport.azurewebsites.net/tla/tla.html [2]: https://en.wikipedia.org/wiki/Communicating_finite-state_mac... [3]: https://en.wikipedia.org/wiki/Petri_net [4]: https://www.adacore.com/sparkpro [5]: https://nim-lang.org/docs/drnim.html [6]: https://github.com/nim-lang/RFCs/issues/222 [7]: https://github.com/mratsim/weave/issues/56 And shameless plug, my talk at the NimConf on debugging multithreading programs (data structure and design): https://www.youtube.com/watch?v=NOAI2wH9Cf0&list=PLxLdEZg8DR... Slides: https://docs.google.com/presentation/d/e/2PACX-1vRq2wvd4q_mo...
In Rust concurrency: the archetype of a message-passing bug
Hey, I've been following up on Legion and Regent quite a bit, excellent work there. Do you have a set of benchmarks that others can reimplement to compare the approaches? I've added Dataflow Parallelism to my own multithreading runtime[1] but I didn't had dataflow focused benchmarks yet, well I could add Cholesky Decomposition but it's quite involved. I expect the people from TaskFlow[2] and Habanero[3] (via Data-Driven Futures) would be quite interested as well in a common set of dataflow parallelism benchmarks. By the way if you didn't read the DaCe paper[4] you absolutely should, seems like the age of Dataflow parallelism and properly optimizing for data is coming. [1]: https://github.com/mratsim/weave#dataflow-parallelism [2]: https://github.com/taskflow/taskflow [3]: https://github.com/habanero-rice/hclib [4]: https://github.com/spcl
Read the rest
In Regent: A Language for Implicit Dataflow Parallelism
As the resident Nim optimizer, if you can reach some performance in C, you can reach it in Nim. With some work you can even reach the performance of code which is 95% assembly and has been tuned for years (i.e. matrix multiplication): - Reaching C Quake code speed in Nim: https://forum.nim-lang.org/t/5124#32241 - Reaching and going beyond OpenBLAS + OpenMP performance (almost pure Assembly): https://github.com/mratsim/weave/issues/68#issuecomment-5717... - Faster than C++ on Raytracing bench: https://github.com/mratsim/weave/tree/9f0c384f/demos/raytrac... - Faster than C++ on another Raytracing implementation: https://github.com/mratsim/trace-of-radiance/tree/v0.2.0 Also Nim is easy to integrate with Intel VTune, Apple Instruments and perf. You have full control on the data layout, prefetching and the same control as C over cache and memo
Read the rest
ry usage if you so wish it (i.e. the GC is not a hindrance, you have easy escape hatches).
In NimConf 2020 Videos
It might be Internet and the issue of communicating emotions across but you sound quite taken by this issue. Anyway, I stand by what I say and I'm backed by my high performance code: - Writing matrix multiplication that is as fast as Assembly, complete with analysis and control on register allocations, L1 and L2 cache tiling and avoiding TLB cache miss: - https://github.com/numforge/laser/blob/master/laser/primitiv... - Code, including caveat about hyperthreading: https://github.com/numforge/laser/blob/master/laser/primitiv... - The code is all pure Nim and is as fast/faster than OpenBLAS when multithreaded, caveat, the single-threaded kernel are slightly slower but it scales better on multiple cores. - I've also written my own multithreading runtime. It's scale better and has lower overhead than Intel TBB. There is no constexpr, you need
Read the rest
type-erasure to handle everything people can use a multithreading runtime for, same comparison on GEMM: https://github.com/mratsim/weave/tree/v0.4.0/benchmarks/matm... - More resources on the importance of memory bandwidth: optimization convolutions https://github.com/numforge/laser/wiki/Convolution-optimisat... - Optimizing matrix multiplication on GPUs: https://github.com/NervanaSystems/maxas/wiki/SGEMM, again it's all about memory and caches optimization - Let's switch to another domain with critical perf need, cryptography. Even when knowing the bounds of iterating on a bigint at compile-time, compiler are very bad at producing optimized code, see GCC vs Clang https://gcc.godbolt.org/z/2h768y - And crypto is the one thing where integer templates are very useful since you know the bounds. - Another domain? VM interpretation. The slowness there is due to function call overhead and/or switch dispatching and not properly using hardware prefetchers. Same thing, C++ constexpr doesn't help it's lower-level, see resources: https://github.com/status-im/nimbus/wiki/Interpreter-optimiz... Also all the polyhedral research, and deep learning compiler research including the Halide compiler, Taichi, Tiramisu, Legion, DaCE confirm that memory is the big bottleneck. Now since you want to stop on the theory and you mentioned HPC, pick your algorithm, it could be matrix multiplication, QR decomposition, Cholesky, ... Any fast C++ code (or C, or Fortran or Assembly) that you find will be fast because of careful memory layout and all level of caches, not constexpr. If you have your own library in one of those domains I would be also very happy to have a look. As a simple example, let's pick an out-of-place transposition kernel to transpose a matrix. Show me how you use constexpr and template metaprogramming to speed it up. Here is a detailed analysis on the impact of 1D-tiling and 2D tiling: https://github.com/numforge/laser/blob/master/benchmarks/tra..., throughput can be increased 4x with proper usage of memory caches.
In A possible new back end for Rust

Showing the 15 most recent of 18 sources. Enable JavaScript to search and browse every indexed mention.

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.