firecracker

From the Firecracker README: https://github.com/firecracker-microvm/firecracker "Firecracker was developed at Amazon Web Services to accelerate the speed and efficiency of services like AWS Lambda and AWS Fargate."

Firecracker has existing solutions to this, which I assume they're using: https://github.com/firecracker-microvm/firecracker/blob/main...

That's the direction container runtimes are trending Amazon has been doing it with Firecracker for a while and Kata containers is another popular one https://github.com/firecracker-microvm/firecracker https://github.com/kata-containers/kata-containers

Thanks for sharing! I'm not sure your table under Why Clone is quite correct. 10x 512MB idle VMs should not take 5GB on Firecracker if they are started from snapshots since the 512MB memory file is mmap'ed with MAP_PRIVATE so is copy on write. Firecracker has diff snapshots: https://github.com/firecracker-microvm/firecracker/blob/main...

firecracker was designed for AWS Lambda and designed to be maintained by multiple AWS teams. You're likely to experience operational + setup hiccups to use it in production. Just look at their setup docs: Setup: https://github.com/firecracker-microvm/firecracker/blob/main... https://github.com/firecracker-microvm/firecracker/blob/main...

Self hosting can be doable for constant small/medium size workloads You can handroll a lot with: https://github.com/nestybox/sysbox?tab=readme-ov-file https://gvisor.dev https://github.com/containers/bubblewrap?tab=readme-ov-file For hardware virtualized machines it much harder but you can do it via: https://github.com/firecracker-microvm/firecracker/ https://github.com/cloud-hypervisor/cloud-hypervisor Freestyle/other providers will likely provide better debugging experience but thats something you can probably get past for a lot of workloads. The time when you/anyone should think about Freestyle/anyone is when the load spikes/the need to create hundreds of VMs in short spikes shows up, or when you're looking for some of the more complex feature sets any given provider has built out (forks, GPUs, network boundaries, etc). I also highly r

It looks like firecracker already supports ACPI vmgenid, which will trigger Linux random to reseed? https://github.com/firecracker-microvm/firecracker/blob/main... https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin... So that just (!) leaves userspace PRNGs.

Good callout. We seed entropy before snapshot to unblock getrandom(), but forks still share CSPRNG state. The proper fix per Firecracker’s docs is RNDADDENTROPY + RNDRESEEDCRNG after each fork, plus reseeding userspace PRNGs like numpy separately. On the roadmap. https://github.com/firecracker-microvm/firecracker/blob/main...

Huh. You're converting FUSE requests into your own custom protocol (with copy-pasted protocol definition) over vsock. Interesting. Not sure I'd trust it with my data[0], but interesting. I don't think the current filepath.Join in realfs.go protects the host against a malicious guest, at all. I'm assuming this is configured as Guest --FUSE--> guest-fused (inside VM) --VSOCK--> realfs. (The Firecracker people have explicitly refused to have virtio-fs, to keep it minimal: https://github.com/firecracker-microvm/firecracker/pull/1351...) https://github.com/jingkaihe/matchlock/blob/123a4df680fb8cc0... https://github.com/jingkaihe/matchlock/blob/123a4df680fb8cc0... https://github.com/jingkaihe/matchlock/blob/123a4df680fb8cc0... [0]: Well, I already know I won't trust hanwen/go-fuse with my data, so that part is a bit moot.

We've only verified EC2 during our research, but you do make a good point here. Nitro wasn't vulnerable. Firecracker might have been, considering that it is also built on top of KVM. Firecracker was not specifically designed to also defend against hardware vulnerabilities [1], so I don't see an immediate reason why it wouldn't have worked. We had to limit the scope of the project somewhere unfortunately, but it would have been nice to check Firecracker and Lambda as well. [1] https://github.com/firecracker-microvm/firecracker/blob/main...

From https://e2b.dev/blog/firecracker-vs-qemu > AWS built [Firecracker (which is built on KVM)] to power Lambda and Fargate [2], where they need to quickly spin up isolated environments for running customer code. Companies like E2B use Firecracker to run AI generated code securily in the cloud, while Fly.io uses it to run lightweight container-like VMs at the edge [4, 5]. "We replaced Firecracker with QEMU" (2023) https://news.ycombinator.com/item?id=36666782 "Firecracker's Kernel Support Policy" describes compatible kernel configurations; https://github.com/firecracker-microvm/firecracker/blob/main... /? wasi microvm kernel [github] https://www.google.com/search?q=wasi+microvm+kernel+GitHub : - "Mewz: Lightweight Execution Environment for WebAssembly with High Isolation and Portability using Unikernels" (2024) https://arxiv.org/abs/241

Yes, that's right. The Firecracker team has written a fantastic doc about this as well: https://github.com/firecracker-microvm/firecracker/blob/main.... It's important to refresh entropy immediately after clone. Still, there can be code that didn't assume it could be cloned (even though there's always been `fork`, of course). Because of this, we don't live clone across workspaces for unlisted/private sandboxes and limit the use case to dev envs where no secrets are stored.