ashvardanian/less_slow.cpp — Hacker News discussions

UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
Period	Start (UTC)	End (exclusive, UTC)	Mentions	Submissions	Comments
Jan 2025	2025-01-01 00:00:00 UTC	2025-02-01 00:00:00 UTC	2	2	0
Feb 2025	2025-02-01 00:00:00 UTC	2025-03-01 00:00:00 UTC	1	0	1
Mar 2025	2025-03-01 00:00:00 UTC	2025-04-01 00:00:00 UTC	2	1	1
Apr 2025	2025-04-01 00:00:00 UTC	2025-05-01 00:00:00 UTC	5	1	4
May 2025	2025-05-01 00:00:00 UTC	2025-06-01 00:00:00 UTC	1	0	1
Jun 2025	2025-06-01 00:00:00 UTC	2025-07-01 00:00:00 UTC	0	0	0
Jul 2025	2025-07-01 00:00:00 UTC	2025-08-01 00:00:00 UTC	0	0	0
Aug 2025	2025-08-01 00:00:00 UTC	2025-09-01 00:00:00 UTC	0	0	0
Sep 2025	2025-09-01 00:00:00 UTC	2025-10-01 00:00:00 UTC	1	0	1

ashvardanian·Sep 23, 2025

Yes, at the scale of 128-bit registers NEON is mostly enough, except for a few categories of instructions missing in that ISA subset, like scatter/gather ops, that can yield 30% boost over serial memory accesses: https://github.com/ashvardanian/less_slow.cpp/releases/tag/v...

In Processing Strings 109x Faster Than Nvidia on H100

ashvardanian·May 6, 2025

Thanks, appreciate the gesture :) Traditional SWAR on GPUs is a fascinating topic. I've begun assembling a set of synthetic benchmarks to compare DP4A vs. DPX (<https://github.com/ashvardanian/less_slow.cpp/pull/35>), but it feels incomplete without SWAR. My working hypothesis is that 64-bit SWAR on properly aligned data could be very useful in GPGPU, though FMA/MIN/MAX operations in that PR might not be the clearest showcase of its strengths. Do you have a better example or use case in mind?

In Faster sorting with SIMD CUDA intrinsics (2024)

aabiji·Apr 20, 2025

Lately I've been going through [less_slow.cpp](https://github.com/ashvardanian/less_slow.cpp) and learning about different performance oriented c++ coding practices.

In Ask HN: What is the most interesting thing you've learned lately?

ashvardanian·Apr 18, 2025

I've made a bunch of bad measurements, until someone reminded me to: #if defined(__AVX512F__) || defined(__AVX2__) void configure_x86_denormals(void) { _MM_SET_FLUSH_ZERO_MODE(_MM_FLUSH_ZERO_ON); // Flush results to zero _MM_SET_DENORMALS_ZERO_MODE(_MM_DENORMALS_ZERO_ON); // Treat denormal inputs as zero } #endif It had a 50x performance impact on Intel. As benchmarked on `c7i.metal-48xl` instances: - `f64` throughput grew from 0.2 to 8.2 TFLOPS. - `f32` throughput grew from 0.6 to 15.1 TFLOPS. Here is that section in the repo with more notes AVX-512, AMX, and other instructions: <https://github.com/ashvardanian/less_slow.cpp/blob/8f32d65cc...>.

In Less Slow C++

ashvardanian·Apr 18, 2025

Most hardware-level observations, like the latency of various memory accesses or numeric operations, would be the same for the Rust code. As for higher-level abstractions, I've already started porting them to Rust <https://github.com/ashvardanian/less_slow.rs>. Next, it would be exciting to implement a concurrent job-stealing graph algorithm in both languages to get a feel for their ergonomics in non-trivial problems. I can imagine it looks very different in Rust and C++, but before I get there, I'm looking for best practices for implementing nested associative containers with shared stateful allocators in Rust. In C++, I've implemented them like this: <https://github.com/ashvardanian/less_slow.cpp/blob/8f32d65cc...>, even though I haven't seen many people doing that in public codebases. Any good examples for Rust?

In Less Slow C++

ashvardanian·Apr 18, 2025

Great question! This has been top of mind for me for the last 2–3 years. Short answer: sadly, no. I love the "usability" promise of coroutines—and even have 2–3 FOSS projects that could be rewritten entirely around C++ or Rust coroutines for better debuggability and extensibility—but my experiments show that the runtime cost of most coroutine‑like abstractions is simply too high. Frankly, I’m not even sure if a better design is possible on modern hardware. This leads me to conclude that, despite my passion for SIMD and superscalar execution, the highest‑impact new assembly instructions that x86 and Arm could standardize would center on async execution and lightweight context switching... yet I haven’t seen any movement in that direction. ⸻ I also wrote toy examples for various range/async/stream models in C++, Rust, and Python, with measu

Read the rest

red latencies in inline comments: - Python: <https://github.com/ashvardanian/less_slow.py/blob/2c80c9dda6ef9aec18264871cfb51a13…> - C++: <https://github.com/ashvardanian/less_slow.cpp/blob/8f32d65cc8976ad20c8cdf645caa5b3…> - Rust: <https://github.com/ashvardanian/less_slow.rs/blob/5add609533854bb6a5ddbf358cc15f06…> Aside from coroutines (toy hand-rolled implementations and commonly used libraries), I've also played around C++ executors, senders & receivers, but didn't have much success with them either. May be a skill issue.

In Less Slow C++

ashvardanian·Apr 18, 2025Submission

Less Slow C++

Earlier this year, I took a month to reexamine my coding habits and rethink some past design choices. I hope to rewrite and improve my FOSS libraries this year, and I needed answers to a few questions first. Perhaps some of these questions will resonate with others in the community, too. - Are coroutines viable for high-performance work? - Should I use SIMD intrinsics for clarity or drop to assembly for easier library distribution? - Has hardware caught up with vectorized scatter/gather in AVX-512 & SVE? - How do secure enclaves & pointer tagging differ on Intel, Arm, & AMD? - What's the throughput gap between CPU and GPU Tensor Cores (TCs)? - How costly are misaligned memory accesses & split-loads, and what gains do non-temporal loads/stores offer? - Which parts of the standard library hit performance hardest? - How do er

Read the rest

ror-handling strategies compare overhead-wise? - What's the compile-time vs. run-time trade-off for lazily evaluated ranges? - What practical, non-trivial use cases exist for meta-programming? - How challenging is Linux Kernel bypass with io_uring vs. POSIX sockets? - How close are we to effectively using Networking TS or heterogeneous Executors in C++? - What are best practices for propagating stateful allocators in nested containers, and which libraries support them? These questions span from micro-kernel optimizations (nanoseconds) to distributed systems (micro/millisecond latencies). Rather than tackling them all in one post, I compiled my explorations into a repository—extending my previous Google Benchmark tutorial (<https://ashvardanian.com/posts/google-benchmark>)—to serve as a sandbox for performance experimentation. Some fun observations: - Compilers now vectorize 3x3x3 and 4x4x4 single/double precision multiplications well! The smaller one is ~60% slower despite 70% fewer operations, outperforming my vanilla SSE/AVX and coming within 10% of AVX-512. - Nvidia TCs vary dramatically across generations in numeric types, throughput, tile shapes, thread synchronization (thread/quad-pair/warp/warp-groups), and operand storage. Post-Volta, manual PTX is often needed (as intrinsics lag), though the new TileIR (introduced at GTC) promises improvements for dense linear algebra kernels. - The AI wave drives CPUs and GPUs to converge in mat-mul throughput & programming complexity. It took me a day to debug TMM register initialization, and SME is equally odd. Sierra Forest packs 288 cores/socket, and AVX10.2 drops 256-bit support for 512-bit... I wonder if discrete Intel GPUs are even needed, given CPU advances? - In common floating-point ranges, scalar sine approximations can be up to 40x faster than standard implementations, even without SIMD. It's a bit hand-wavy, though; I wish more projects documented error bounds and had 1 & 3.5 ULP variants like Sleef. - Meta-programming tools like CTRE can outperform typical RegEx engines by 5x and simplify building parsers compared to hand-crafted FSMs. - Once clearly distinct in complexity and performance (DPDK/SPDK vs. io_uring), the gap is narrowing. While pre-5.5 io_uring can boost UDP throughput by 4x on loopback IO, newer zero-copy and concurrency optimizations remain challenging. The repository is loaded with links to favorite CppCon lectures, GitHub snippets, and tech blog posts. Recognizing that many high-level concepts are handled differently across languages, I've also started porting examples to Rust & Python in separate repos. Coroutines look bad everywhere :( Overall, this research project was rewarding! Most questions found answers in code — except pointer tagging and secure enclaves, which still elude me in public cloud. I'd love to hear from others, especially on comparing High-Level Synthesis for small matrix multiplications on FPGAs versus hand-written VHDL/Verilog for integral types. Let me know if you have ideas for other cool, obscure topics to cover!

198 points · Open discussion

ashvardanian·Mar 31, 2025Submission

Show HN: Less Slow C++

Earlier this year, I took a month to reexamine my coding habits and rethink some past design choices. I hope to rewrite and improve my FOSS libraries this year, and I needed answers to a few questions first. Perhaps some of these questions will resonate with others in the community, too. - Are coroutines viable for high-performance work? - Should I use SIMD intrinsics for clarity or drop to assembly for easier library distribution? - Has hardware caught up with vectorized scatter/gather in AVX-512 & SVE? - How do secure enclaves & pointer tagging differ on Intel, Arm, & AMD? - What's the throughput gap between CPU and GPU Tensor Cores (TCs)? - How costly are misaligned memory accesses & split-loads, and what gains do non-temporal loads/stores offer? - Which parts of the standard library hit performance hardest? - How do er

Read the rest

ror-handling strategies compare overhead-wise? - What's the compile-time vs. run-time trade-off for lazily evaluated ranges? - What practical, non-trivial use cases exist for meta-programming? - How challenging is Linux Kernel bypass with io_uring vs. POSIX sockets? - How close are we to effectively using Networking TS or heterogeneous Executors in C++? - What are best practices for propagating stateful allocators in nested containers, and which libraries support them? These questions span from micro-kernel optimizations (nanoseconds) to distributed systems (micro/millisecond latencies). Rather than tackling them all in one post, I compiled my explorations into a repository—extending my previous Google Benchmark tutorial (<https://ashvardanian.com/posts/google-benchmark>)—to serve as a sandbox for performance experimentation. Some fun observations: - Compilers now vectorize 3x3x3 and 4x4x4 single/double precision multiplications well! The smaller one is ~60% slower despite 70% fewer operations, outperforming my vanilla SSE/AVX and coming within 10% of AVX-512. - Nvidia TCs vary dramatically across generations in numeric types, throughput, tile shapes, thread synchronization (thread/quad-pair/warp/warp-groups), and operand storage. Post-Volta, manual PTX is often needed (as intrinsics lag), though the new TileIR (introduced at GTC) promises improvements for dense linear algebra kernels. - The AI wave drives CPUs and GPUs to converge in mat-mul throughput & programming complexity. It took me a day to debug TMM register initialization, and SME is equally odd. Sierra Forest packs 288 cores/socket, and AVX10.2 drops 256-bit support for 512-bit... I wonder if discrete Intel GPUs are even needed, given CPU advances? - In common floating-point ranges, scalar sine approximations can be up to 40x faster than standard implementations, even without SIMD. It's a bit hand-wavy, though; I wish more projects documented error bounds and had 1 & 3.5 ULP variants like Sleef. - Meta-programming tools like CTRE can outperform typical RegEx engines by 5x and simplify building parsers compared to hand-crafted FSMs. - Once clearly distinct in complexity and performance (DPDK/SPDK vs. io_uring), the gap is narrowing. While pre-5.5 io_uring can boost UDP throughput by 4x on loopback IO, newer zero-copy and concurrency optimizations remain challenging. The repository is loaded with links to favorite CppCon lectures, GitHub snippets, and tech blog posts. Recognizing that many high-level concepts are handled differently across languages, I've also started porting examples to Rust & Python in separate repos. Coroutines look bad everywhere :( Overall, this research project was rewarding! Most questions found answers in code — except pointer tagging and secure enclaves, which still elude me in public cloud. I'd love to hear from others, especially on comparing High-Level Synthesis for small matrix multiplications on FPGAs versus hand-written VHDL/Verilog for integral types. Let me know if you have ideas for other cool, obscure topics to cover!

6 points · Open discussion

ashvardanian·Mar 30, 2025

… except for byte-level processing, variable-length codecs, or mixed-precision numerics. That never works with autovectorization and can’t be solved with general-purpose SIMD wrappers. For me the solution was to implement those manually, and even at a scale of just 2 libraries I’ve eneded up with somewhat different project layouts & dispatch mechanisms: https://github.com/ashvardanian/SimSIMD , https://github.com/ashvardanian/StringZilla One big family not covered there, is sparse data-strictures and related algorithms. I’ve only started integrating scatter/gather in AVX-512 and SVE, and on synthetic benchmarks both look promising: https://github.com/ashvardanian/less_slow.cpp/releases/tag/v... Those should probably unlock a much wider set of applications for SIMD, but designing libraries for those may benefit from yet another project str

Read the rest

ucture.

In Towards fearless SIMD, 7 years later

ashvardanian·Feb 27, 2025

I generally avoid FP8 and prefer I8, but your question got me wondering how well cuBLAS performs. First of all, cuBLAS needs the cuBLASLt extension API for mixed-precision workloads to handle FP8. Second, some adequate type combinations, like E5M2 x E5M2 for A x B, are not supported, while others, like E5M2 x E4M3, are! Moreover, matrix A must always come in a transposed layout for Ampere, Hopper, and Blackwell... and the list of constraints goes on. I've integrated FP8 cuBLASLt benchmarks into my "Less Slow C++" repository <https://github.com/ashvardanian/less_slow.cpp>, adding to the list of existing cuBLAS and hand-rolled CUDA and PTX benchmarks. I'm running them on H200 GPUs, which should have the same performance as H100. For square inputs, the throughput peaks around 1.35 Peta-ops. -------------------------------------------------

Read the rest

------------------------------------------------- Benchmark Time CPU Iterations UserCounters... -------------------------------------------------------------------------------------------------- cublaslt_tops<fp8_e4m3_t, float>/256 12496 ns 12496 ns 56284 TOP=2.67999T/s cublaslt_tops<fp8_e4m3_t, float>/512 13089 ns 13089 ns 53100 TOP=20.4883T/s cublaslt_tops<fp8_e4m3_t, float>/1024 14882 ns 14882 ns 46918 TOP=144.23T/s cublaslt_tops<fp8_e4m3_t, float>/2048 25802 ns 25802 ns 26869 TOP=665.679T/s cublaslt_tops<fp8_e4m3_t, float>/4096 109316 ns 109313 ns 6021 TOP=1.25715P/s cublaslt_tops<fp8_e4m3_t, float>/8192 821080 ns 821050 ns 629 TOP=1.33907P/s cublaslt_tops<fp8_e4m3_t, float>/16384 7135472 ns 7135461 ns 93 TOP=1.23269P/s cublaslt_tops<fp8_e4m3_t, float>_BigO 0.00 N^3 0.00 N^3 cublaslt_tops<fp8_e4m3_t, float>_RMS 2 % 2 % That's around 67% of the advertised number for dense GEMM <https://resources.nvidia.com/en-us-data-center-overview-mc/e...>.

In DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling

thunderbong·Jan 29, 2025Submission

Release Less Slow CPP v0.6: Thrust → CUDA → PTX → SASS

3 points · Open discussion

asicsp·Jan 4, 2025Submission

Learning to Write Less Slow C, C++, and Assembly Code

15 points · Open discussion