alive2

Undefined behavior does not violate correctness. Undefined behavior is just wiggle room for compiler engineers to not have to worry so much about certain edge cases. "Correctness" must always be considered with respect to something else. If we take e.g. the C specification, then yes, there are plenty of compilers that are in almost all ways people will encounter correct according to that spec, UB and all. Yes, there are bugs but they are bugs and they can be fixed. The LLVM project has a very neat tool called Alive2 [1] that can verify optimization passes for correctness. I think there's a very big gap between the kind of reliability we can expect from a deterministic, verified compiler and the approximating behavior of a probabilistic LLM. [1]: https://github.com/AliveToolkit/alive2

But we do! For llvm there's https://github.com/AliveToolkit/alive2 There are papers like https://people.cs.rutgers.edu/~sn349/papers/cgo19-casmverify... There's https://github.com/google/souper There's https://cr.yp.to/papers/symexemu-20250505.pdf And probably other things I'm not aware of. If you're limiting the scope to a few blocks at a time, symbolic execution will do fine.

Another problem with LLVM I’ve heard about is that it’s intermediate language or API or something is a moving, informally-specified target. People who know LLVM internals might weigh in on that claim. If true, it’s actually easier to target C or a subset of Rust just because it’s static and well-understood. Two projects sought to mitigate these issues by going in different directions. One was a compiler backend that aimed to be easy to learn with well-specified IL. The other aimed to formalize LLVM’s IL. http://c9x.me/compile/ https://github.com/AliveToolkit/alive2 There have also been typed, assembly languages to support verification from groups like FLINT. One can also compile language-specific analysis with a certified to LLVM IL compiler. Integrating pieces from different languages can have risks. That (IIRC) is being mitigated by peo

This is very cool, especially the automatic proof generation! That said, I wonder if it would make more sense to use an SMT prover as fallback rather than a SAT solver. The latter requires converting the 32-bit operations into boolean circuits, which might not be the most efficient way to solve the problem considering modern SMT solvers such as Z3 and CVC5 already have built-in efficient support for bit-vectors? In fact, I believe that's what Alive2 [1] does, which is a similar tool (although with slightly different goals). I think it might even be possible to make this work for 64-bit vectors with SMT (in most cases). [1] https://github.com/AliveToolkit/alive2

You can use it to (mostly) validate small snippets are the same. See Alive2 for the application of Z3/formalization of programs as SMT for that [1]. As far as I'm aware there are some problems scaling up to arbitrarily sized programs due to a lack of formalization in higher level languages in addition to computational constraints. With a lot of time and effort it can be done though [2]. 1. https://github.com/AliveToolkit/alive2 2. https://sel4.systems/

Yeah, this kind of thing is nice. Alive had been used for years (almost a decade actually) by people to verify LLVM instcombine transforms. Alive2 (https://github.com/AliveToolkit/alive2) makes it easier to do the same with most optimization passes.

Alive/Alive2 [1] is one of the most famous frameworks for compiler transformation verification using BitVec logic [1] https://github.com/AliveToolkit/alive2