xbyak

I'm going to hijack the mention of ML to share xbyaku, a c++ library presenting a DSL for assembling machine code at runtime (useful for JIT). It's used by some of the pytorch back ends. https://github.com/herumi/xbyak Example use: https://github.com/oneapi-src/oneDNN/blob/main/src/cpu/aarch... I learned about these through a blog post about speeding up pytorch on ARM: https://pytorch.org/blog/optimized-pytorch-w-graviton/

Note that libdivide may sound like an improvement, however there are better solutions today: 1) full-blown compiler/engines like LLVM or GCC build close to optimal shift/mul-based code (without overhead for simple constants like "2") 2) if one can't include LLVM, there are lightweight libraries like xbyak or asmjit, where one can implement optimal jit solution. See https://github.com/herumi/xbyak/blob/80477f635345e8f13efc512... as a starting point.

About 6 months ago my job required that I finally get my hands dirty writing x86 assembly. It's my first real foray into assembly coding. There are a few aspects of it that I'm really enjoying: - I can now actually understand the disassembled code that I see during debugging. This includes recognizing some of the assembly patterns that appear because of ABI requirements and/or common programming idioms. - I'm becoming comfortable with a programming idiom that I've never really used in the past: registers, flags, various kinds of memory addressing. - It helps my understanding of compilers' lower levels / backends, and the related problems: register allocation, instruction selection, etc. - It provides a clear path for my first attempt at writing JIT code (using Xbyak[0]). So as Richard Feynman might have said, it's great fun! [0] https://g

I'm still a bit of an assembly/JIT newbie, but a few thoughts: When you write assembly code, it's generally specific to several details, which in turn affects your choice of tutorials from which to work: 1) The target architecture (x86-64, AArch64, MIPS, etc.) This defines what instructions you have available, the memory model, etc. 2) The architecture ABI. E.g., what's the protocol for passing parameters to functions you call, and for receiving the return code. [0] 3) The particular assembler you plan to use. Gnu As (and I assume others) provide some directives that don't map directly to machine instructions, but do some book-keeping to make your life easier. 4) (Potentially) the file format for the resulting code [1]. I think this is one area where the assembler and linker utilities can shelter you from the ugly details. FWIW regarding

An Intel engineer sends PR https://github.com/herumi/xbyak/pull/95

Is there anything like https://github.com/herumi/xbyak on the Rust side? I like this approach a lot for non-tivial high performance work since it lets you tweak your assembly for particular hardware at runtime.

AMD needs to do something like https://github.com/intel/mkl-dnn and https://github.com/opencv/dldt, either by talking to Intel to get them to accept contributions (vastly preferable), or, failing that, by forking the libs and implementing their own support. With recent work on efficient NN architectures the CPUs are pretty viable for most _inference_ (not training) tasks. DLDT/OpenVINO perf is particularly impressive. It does, however, extensively use JIT kernel generation through https://github.com/herumi/xbyak depending on the supported features for a particular chip (search for "mayiuse()" in the code to see examples). It's anyone's guess what it will detect and generate on an AMD chip however. Now that AMD is raking in more cash, it'd be a great time to invest some of that back into their dev ecosystem.

This could be pretty cool. Right now you have to drop down to non-portable, assembly-style JIT like Xbyak (https://github.com/herumi/xbyak) or VIXL on ARM (https://community.arm.com/developer/ip-products/processors/b...). It might not be obvious why you'd want something like that, so I'll explain. Basically think of this as templates which you don't have to pre-instantiate. You can specify exactly how long your loops are, so the compiler will be able to do a much better job of vectorizing, eliminating branches, etc etc. In a tight math-kernel style code this could easily improve the perf by 2x or more. Combined with intrinsics this could eliminate the need for assembly in many cases. This would be exciting if it came to fruition at some point. Unfortunately C++ standard being what it is, we're looking at like 2025 before we see the first

Very nice, starred. There's also xbyak, which a number of projects use: https://github.com/herumi/xbyak. I just wish something like this was available for ARM. There's xtaak (ARM port of xbyak), but it's woefully incomplete.

I'm playing a lot with VM, interpreters, JIT and recently compilers in Nim. Here is a JIT brainfuck VM: https://github.com/mratsim/jitterland Nim macros are extremely useful for assemblers and emulators as contrary to many codebases which needs to use an intermediate code generation step via text files [1][2][3] you can directly use them with type checking and all Nim facilities while keeping an easy to read syntax: https://github.com/numforge/laser/blob/e898f027/laser/photon... [1]: Xbyak - https://github.com/herumi/xbyak/blob/v5.751/gen/gen_avx512.c... [2]: NNPack + PeachPy - https://github.com/Maratyszcza/NNPACK/blob/af40ea7d/src/x86_... [3]: Apache Milagro Crypto - https://github.com/apache/incubator-milagro-crypto/blob/d43a...

A while ago, I spent a fair amount of time evaluating JIT frameworks including DynASM. The one I ended up using is something called Xbyak (https://github.com/herumi/xbyak) which I think is a bit of a hidden gem. Unlike DynASM, it doesn't require a preprocessing pass, is better documented and IMHO easier to use. So if you're in the market for a lightweight JIT engine and target C++/X86, I'd give Xbyak a whirl too and see which one you like better.