HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
mratsim
laser
Nim2018–2024
mratsim / laser

laser

The HPC toolbox: fused matrix multiplication, convolution, data-parallel strided tensor primitives, OpenMP facilities, SIMD, JIT Assembler, CPU detection, state-of-the-art vectorized BLAS for floats and integers

4Distinct people
21Mentions
0Explicit recommenders
295GitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

The HPC toolbox: fused matrix multiplication, convolution, data-parallel strided tensor primitives, OpenMP facilities, SIMD, JIT Assembler, CPU detection, state-of-the-art vectorized BLAS for floats and integers

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

Mentioned in the same Hacker News threads. That is not proof they are substitutes.

This week on Hacker News

Nov 9–Nov 16 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/16/2026, 1:56:48 AM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year21 dated mentions

Observed mentions: 2018-12-19 17:48:10 UTC to 2024-07-20 19:28:21 UTC.

024682018-01-01 00:00:00 UTC to 2019-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments20182019-01-01 00:00:00 UTC to 2020-01-01 00:00:00 UTC (exclusive): 7 mentions; 0 submissions, 7 comments2020-01-01 00:00:00 UTC to 2021-01-01 00:00:00 UTC (exclusive): 8 mentions; 0 submissions, 8 comments2021-01-01 00:00:00 UTC to 2022-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments20212022-01-01 00:00:00 UTC to 2023-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-01-01 00:00:00 UTC to 2024-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-01-01 00:00:00 UTC to 2025-01-01 00:00:00 UTC (exclusive): 5 mentions; 0 submissions, 5 comments2024

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
20182018-01-01 00:00:00 UTC2019-01-01 00:00:00 UTC101
20192019-01-01 00:00:00 UTC2020-01-01 00:00:00 UTC707
20202020-01-01 00:00:00 UTC2021-01-01 00:00:00 UTC808
20212021-01-01 00:00:00 UTC2022-01-01 00:00:00 UTC000
20222022-01-01 00:00:00 UTC2023-01-01 00:00:00 UTC000
20232023-01-01 00:00:00 UTC2024-01-01 00:00:00 UTC000
20242024-01-01 00:00:00 UTC2025-01-01 00:00:00 UTC505

What Hacker News says

7 sources
The follow-up paper should include the deep learning and linear algebra compilers: - Halide - Tiramisu - Ngraph - Weld - PyTorch JIT & PyTorch Glow and Tensor Comprehensions - TVM - Futhark - A toy MLIR language build on the linalg dialect. I'm sure Google would love to help them. to name the main ones. Everyone is moving towards a compiler approach. I'm actively researching the domain and I think Halide is the most promising: by separating the algorithm from the schedule you can leave the algorithm side to the researchers and optimization is done by HPC devs or automated methods without having to rewrite in C/Fortran. Much more attempts in the past: - https://github.com/mratsim/Arraymancer/issues/347#issuecomme... - My own domain specific language and compiler: https://github.com/numforge/laser/tree/master/laser/lux_comp... there are a
Read the rest
low_level_design_considerations.md and challenges.md that should be interesting as well. Unfortunately I didn't go beyond the proof of concept as I need to revamp the Nim multithreading runtime to improve composition of parallel linear algebra kernels. One thing for sure is that OpenMP is today something that needs to be workaround due to it's lack of composability and GCC implementation is a contention point due to using a single queue protected by a lock. While traditional linear algebra algorithms can deal with static distribution and no load balancing, some algorithms would hugely benefits from a more flexible task system, for example Beam Search.
In The Linear Algebra Mapping Problem
Yes exactly. Usually you can extract 2x to 4x instruction level parallelism on simple add instructions [1] vs [2] For fused-multiply-add even though the latency is higher the instruction is also slower so its the same. BLIS, OpenBLAS and my code extract 2x parallelism (2 accumulators) at the lowest level because we are restricted by the number of registers and the fact that x86 can only issue 2 FMAs per cycle [3]. Details, we divided the work until we have a small micro matrix C or size MR x NR (input matrix of size MxK and KxN so output of MxN), and then here is are the constraints you have to deal with: Registers constraints and micro-kernel tuning - To issue 2xFMAs in parallel we need to use 2x SIMD registers - We want to hold C of size MR * NR completely in SIMD registers as well as each value is reused k times during accumula
Read the rest
tion C[i, j] += A[i, k] * B[k, j] - We should have enough SIMD registers left to hold the corresponding sections of A and B (at least 4, 2xA and 2xB for FMAs) On x86-64 X SIMD registers that can issue 2xFMAs per cycle: - NbVecs is 2 minimum - RegsPerVec = 2 * NbVecs => 4 minimum (for A and for B) - NR = NbVecs * NbScalarsPerSIMD - C: MR*NR and uses MR*NbVecs SIMD registers - MR*NbVecs + RegsPerVec <= X -> MR*NbVecs + 2 * NbVecs <= X -> (MR+2) * NbVecs <= X Some solutions: - AVX with 16 registers: - MR = 6, NbVecs = 2 FP32: 8xFP32 per SIMD --> NR = 2x8 ukernel = 6x16 FP64, ukernel = 6x8 - MR = 2, NbVecs = 4 FP32: 8xFP32 per SIMD --> NR = 4x8 ukernel = 2x32 FP64, ukernel = 2x16 - AVX512 with 32 registers - MR = 6, NbVecs = 4 FP32 ukernel = 6x64 FP64 ukernel = 6x32 - MR = 2, NbVecs = 8 FP32 ukernel = 2x128 FP64 ukernel = 2x64 - MR = 14, NbVecs = 2 FP32 ukernel = 14x32 FP64 ukernel = 14x16 And in-depth overview of the lowest level details is available in the paper Automating the last mile for High Performance Dense Linear Algebra[5]. In short, the compiler is completely unable to deal with this, and a high performance computing compiler should give an escape hatch to allow hand optimization like Halide[6] or Tiramisu do[7]. [1]: https://github.com/numforge/laser/blob/e660eeeb723426e80a7b1... [2]: https://github.com/numforge/laser/blob/e660eeeb723426e80a7b1... [3]: https://github.com/numforge/laser/blob/e660eeeb723426e80a7b1... [4]: https://github.com/numforge/laser/blob/e660eeeb723426e80a7b1... [5]: https://arxiv.org/pdf/1611.08035.pdf [6]: https://halide-lang.org/ [7]: http://tiramisu-compiler.org/
In Comparing Parallel Rust and C++
If you are interested on the exact same topics (matrix multiplication parallelization) here are other step by step tutorials I used: - UlmBLAS [1], for a HPC course in 14 steps - BlisLab [2], make sure to checkout the tutorial.pdf. It gives you exercises to solve in C and each one build on the previous one In Rust matrixmultiply crate [3] implements those techniques to reach 80% of OpenBLAS speed, see blog post GEMM: a rabbit hole [4]. This is a generic approach that can be followed by any low-level languages. I reach 100% of OpenBLAS and MKL-DNN speed in Nim on large matrices as well [5] without any assembly and a generic code that can also generates integer matrix multiply [6]. Regarding fast-math, that's what you do manually, you interleave the fused-multiply adds as they have a latency of about 6 cycles (Broadwell, I don't remember on
Read the rest
In Comparing Parallel Rust and C++
If a language allows you: - to map cleanly with what compilers know to optimize best - fine control over stack and heap allocation - has intrinsics / assembly escape hatch - allows you to specify that pointers don't alias (restrict in C, or default in Fortran) - gives you prefetching primitives You will be able to reach hand-tuned Assembly-like performance (and not just C-like performance). Case in-point, I tuned my own matrix multiplication algorithms in Nim to carefully control register allocations, L1, L2 and L3 cache usage, and vector intrinsics to reach the speed of assembly tuned OpenBLAS and Intel MKL-DNN (no assembly at all): bench: https://github.com/numforge/laser/blob/c7ddceb0/benchmarks/g... code: https://github.com/numforge/laser/tree/c7ddceb0/laser/primit... And matrix multiplication has decades of research and now dedicated
Read the rest
hardware (tensor cores, EPU, TPU, NPU, ...) as this is a key algorithm for most numerical workloads.
In Zig 0.5.0 Release Notes
Personally, I use Nim to develop domain specific language for VM, JIT, emulators and neural networks. I'm currently writing a compiler for deep learning with both AOT (emitting Nim code) and hopefully later JIT (emitting LLVM IR) capabilities complete with SIMD support. I don't see how I could do that in another language. My biggest successes: - include a quite maintainable JIT for x86_64 (compared to asmjit and xbyak I don't need to parse or codegen the C++ code): https://github.com/numforge/laser/blob/master/laser/photon_j... - a DSL for neural network: https://github.com/mratsim/Arraymancer#sequence-classificati... - a matrix multiplication BLAS written from scratch competitive with OpenBLAS and MKL on select matrix shape (2000x2000) but that can also support int8/int16/int32/int64 and not just float32/float64 thanks to metaprogramming
Read the rest
. Expanding to new SIMD architecture (ARM) is very easy: --> benchmark: https://github.com/numforge/laser/blob/master/benchmarks/gem... --> Metaprogramming AVX512 support: https://github.com/numforge/laser/blob/master/laser/primitiv... The most important thing for me for metaprogramming is being able to operate on the AST directly
In Ask HN: What metaprogrammable language do you/would like to use?
I'm playing a lot with VM, interpreters, JIT and recently compilers in Nim. Here is a JIT brainfuck VM: https://github.com/mratsim/jitterland Nim macros are extremely useful for assemblers and emulators as contrary to many codebases which needs to use an intermediate code generation step via text files [1][2][3] you can directly use them with type checking and all Nim facilities while keeping an easy to read syntax: https://github.com/numforge/laser/blob/e898f027/laser/photon... [1]: Xbyak - https://github.com/herumi/xbyak/blob/v5.751/gen/gen_avx512.c... [2]: NNPack + PeachPy - https://github.com/Maratyszcza/NNPACK/blob/af40ea7d/src/x86_... [3]: Apache Milagro Crypto - https://github.com/apache/incubator-milagro-crypto/blob/d43a...
In Show HN: Synesthesia – Optimizing brainfuck compiler implemented as Nim macros

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.