live-bootstrap

Worth checking out https://github.com/fosslinux/live-bootstrap which starts from a handwritten hex bootloader and later a Scheme interpreter that implements a C compiler that compiles tcc Full steps on https://github.com/fosslinux/live-bootstrap/blob/master/part...

That's not what I said. I've implied that it's hard to trust the output of some unknown compiler (eg, the "zig1.wasm" blob) and that it's easier to trust source code. The Dozer article explains, under "The Descent Principle", how rustc will eventually be buildable using only source code [0] (other than a "512-byte binary seed" which implements a trivial hex interpreter). You still need to trust a computer to run everything on, though in theory it should be possible to gain trust by running it on multiple computers and checking that the result is the same (this is why any useful system bootstrapping project should also be reproducible [1]). [0] https://github.com/fosslinux/live-bootstrap [1] https://bootstrappable.org/best-practices.html

Yeah pretty much, the best example I have found for showcasing a solution to this issue is this[1] example, where you can see we start from the most basic of "compilers" (quite literally the equivalent to a `sed` command) and work our way up to Linux 4.9 if I remember correct. Biggest issue is circular dependencies (a lot of lower level build tools depend on themselves nowadays, so we end up needing to build like 4-5 older versions to work our way to a modern toolchain) and different architectures. Since x86 has always been a given, having a completely bootstrappable toolchain on something like arm or even risc-v is a more complex problem, where older versions of programs needed may not necessarily compile or be able to compile further programs on these architectures. [1]: https://github.com/fosslinux/live-bootstrap

>From there they can bootstrap yacc, basic coreutils, Bash, autotools, and eventually GCC ... it’s a fascinating process. I would say about half of the list can be trimmed off if you managed to separate GCC 4 and binutils from their original build scripts, notice the sheer amounts of items there are just repeatedly rebuilding auto-stuff and their dependencies[1]. [1] https://github.com/fosslinux/live-bootstrap/blob/master/part...

You can now have trustworthy Rust compiler binaries, through the work of the Bootstrappable Builds community, which found a way to build a C compiler without having C compiler binaries yet. https://bootstrappable.org/ https://github.com/fosslinux/live-bootstrap/

Interesting read. I have a lot of respect for people who develop emulator for x86 processors. It is a complicated processor and from first hand experience I know that developing and debugging emulators for CPU's can be very challenging. In the past year, I spend some time developing a very limited i386 emulator [1] including some system calls for executing the first steps of live-bootstrap [2], primarily to figure out how it is working. I learned a lot about system calls and ELF. [1] https://github.com/FransFaase/Emulator/ [2] https://github.com/fosslinux/live-bootstrap/

> (...) rebuilding packages using a bootsrappable build process, both seem orthogonal to the idempotent rebuild problem You know what would be awesome? If someone could start from, let's say, live-bootstrap[1] and build towards matching the checksums for some distro kernel+toolchain. It sounds like the same kind of problem, it all comes down to knowing what build conditions affect the resulting binaries, so I think you nailed the problem description on this and yes, it all feels very orthogonal from that perspective! Thanks for writing this blog entry! [1]: http://github.com/fosslinux/live-bootstrap

Yes! You can see the full path here: https://github.com/fosslinux/live-bootstrap/blob/master/part...

I believe so! I did some messing around with the whole "bootstrappable" suite of repos a few months ago and I remember there was a repo [1] that automated the chain of bootstrapping from a project known as hex0 to Linux 4.9 (iirc) inside qemu using fiwix as an intermediary. I didn't have the time to experiment past running it and verifying it works (it did, took my poor laptop around 10 or so hours to run from start to finish), but I presume I would have been able to compile the latest versions of GCC and Linux from the final state of the VM it made. I may still have the image it made lying around somewhere. [1]: https://github.com/fosslinux/live-bootstrap

strace is a rather powerfull tool if you want to find out what a certain executable is doing. Which files it is opening, reading and writing and also which other executables it is executing. I personally have not used the '--stack-trace' option yet. Earlier this year, I have used it to analyze what happens during the initial steps of live-bootstrap [1] and produce a web page with all the information [2]. For this, I wrote a C program to parse and process the output of strace. [1] https://github.com/fosslinux/live-bootstrap [2] https://fransfaase.github.io/Emulator/

Look at how to bootstrap a compiler, kernel etc from source when you don't have any binaries for compilers, kernels etc. https://bootstrappable.org/ https://github.com/fosslinux/live-bootstrap/ https://bootstrapping.miraheze.org/wiki/Stage0