ml-compiler-opt

I did a bit of work on this last summer on (much) smaller models [1] and it was briefly discussed towards the end of last year's MLGO panel [2]. For heuristic replacements specifically, you might be able to glean some things (or just use interpretable models like decision trees), but something like a neural network works fundamentally differently than the existing heuristics, so you probably wouldn't see most of the performance gains. For just tuning heuristics, the usual practice is to make most of the parameters configurable and then use something like bayesian optimization to try and find an optimal set, and this is sometimes done as a baseline in pieces of ML-in-compiler research. 1. https://github.com/google/ml-compiler-opt/pull/109 2. https://youtu.be/0uUKDQyn1Z4?si=PHrx9RICJIiA3E6C

If you're using Clang/LLVM you can also enable ML inlining[1] (assuming you build from source) which can save up to around 7% if all goes well. There are also talks of work on just brute forcing the inlining for size problem for embedded releases for smallish applications. It's definitely feasible if the problem is important enough to you to throw some compute at it [2]. 1. https://github.com/google/ml-compiler-opt 2. https://doi.org/10.1145/3503222.3507744

Looks like they do have a pretrained model: https://github.com/google/ml-compiler-opt/releases/tag/inlin... The code will by default auto-download it during the build process. It's about 800 kbytes, which seems very reasonable for something that will reduce the generated code size by gigabytes for a large codebase. Note that the open-source-ness is dubious... They say it was trained using an optimizer which isn't opensource, and that the results are significantly better ('more generalizable') than using the open source one. In my view, if the code that makes a binary blob isn't opensource, then the project isn't opensource...

Based on https://github.com/google/ml-compiler-opt#pretrained-models, it seems like the models are versioned, so I guess they'd update their models when updating LLVM or something like that.

I'm not sure - https://github.com/google/ml-compiler-opt#pretrained-models says the they sometimes get released on GitHub but I couldn't see anything obvious in that repo.

They trained (and report) two optimization strategies: - inline-for-size We trained the inlining-for-size policy on a large internal software package containing 30k modules. The trained policy is generalizable when applied to compile other software and achieves a 3% ~ 7% size reduction. - regalloc with 0.3% ~1.5% improvements in queries per second (QPS) on a set of internal large-scale datacenter applications Try it Yourself Check out the open-sourced end-to-end data collection and training solution on github and a demo that uses policy gradient to train an inlining-for-size policy. https://github.com/google/ml-compiler-opt https://github.com/google/ml-compiler-opt/blob/main/docs/demo/demo.md With code, that's awesome—what I like to see.