remote-apis

I'm referring to this: https://github.com/bazelbuild/remote-apis Once you get a very large C++ project with several thousand compilation jobs over hundreds of devs, you need to distribute the build across multiple computers and have a shared cache for object files. Zig doesn't seem to support that.

I've been wondering idly if it's possible for Nix to support the Bazel Remote Execution API that seems to be catching on[1] more generally. [1] https://github.com/bazelbuild/remote-apis?tab=readme-ov-file...

In the case of the Remote Execution/Cache API used by Bazel among others[1] at least, it's a bit more detailed. There's an "ActionCache" and an actual content-addressed cache that just stores blobs ("ContentAddressableStorage"). When you run a `gcc -O2 foo.c -o foo.o` command (locally or remotely; doesn't matter), you upload an "Action" into the action cache, which basically said "This command was run. As a result it had this stderr, stdout, error code, and these input files read and output files written." The input and output files are referenced by the hash of their contents, in this case, and they get uploaded into the CAS system. Most importantly you can look up an action in the ActionCache without actually running it, provided you have the inputs at hand. So now when another person comes by and runs the same build command, they say "

Not only it's distributed like distcc, Bazel also provide sandboxing to ensure that environment factors does not affect the build/test results. This might not mean much for smaller use cases, but at scale with different compiler toolchains targeting different OS and CPU Architecture, the sandbox helps a ton in keeping your build result accurate so that it could be cached remotely (like using ccache). On top of it, the APIs Bazel uses to communicate with the remote execution environment is standardized and adopted by other build tools with multiple server implementation to match it. Looking into https://github.com/bazelbuild/remote-apis/#clients, you could see big players are involved: Meta, Twitter, Chromium project, Bloomberg while there are commercial supports for some server implementations. Finally, on top of C/C++, Bazel also support

I would suggest also looking at https://github.com/bazelbuild/remote-apis. Its essentially a standard API for remote (any binary) execution as a service and there are several reference implementations of it (Buildgrid, BuildBarn, Google's own service etc). And you can consider using gVisor to minimize container breakouts to a great extent.

Regardless of whether you should use Bazel or not, my hope is that any future build systems attempt to adopt Bazel's remote execution protocol (or at least a protocol that is similar in spirit): https://github.com/bazelbuild/remote-apis In my opinion the protocol is fairly well designed.

> The thing I really would like to see is a smarter CI system. Caching of build outputs, so you don't have to rebuild the world from scratch every time. Distributed execution of tests and compilation, so you are not bottle-necked by one machine. This is already achievable nowadays using Bazel (https://bazel.build) as a build system. It uses a gRPC based protocol for offloading/caching the actual build on a build cluster (https://github.com/bazelbuild/remote-apis). I am the author of one of the Open Source build cluster implementations (Buildbarn).

Very nice! I really like the ease-of-use of this, as well as the scale-to-zero costs. That's a tricky thing to achieve. Seems like it could become a standard path to ease the migration from local to remote builds. If the author is interested in standardizing the same, I'd suggest implementing the REAPI protocol (https://github.com/bazelbuild/remote-apis). It should be amenable to implementing on a Lambda-esque back-end, and is already standard amongst most tools doing Remote Execution (including Bazel! Bazel+llama could be fun). And equally, it's totally usable by a distcc-esque distribution tool (recc[1] is one example) - that's also what Android is doing before they finish migrating to Bazel ([2], sadly not yet oss'd). The main interesting challenge I expect this project to hit is going to be worker-local caching: for compilation action

> 5) Forge -- basically Blaze's remote build execution (what Bazel calls RBEs). Almost every build at Google is extremely parallelized, and if you need to run flake tests, running a suite of tests 10,000 times is almost as fast as running it once. This is available outside Google now - start at https://github.com/bazelbuild/remote-apis or https://docs.bazel.build/versions/master/remote-execution.ht... . It's a standardized API, supported by a growing set of build tools (notably Bazel, Pants, Please) with a variety of OSS and Commercial implementations. At this point almost anyone can set up Remote Execution if they wish to, and remote caching is even easier. Minor terminology correction: RBE generally refers to Google's own implementation of the same name; Remote Execution (RE) and the REAPI are used to refer to the generic concept. (Disc