bb-remote-execution

Seems to be nfs v3 [0] - curious to test it out - the only userspace nfsv4 implementation I’m aware of is in buildbarn (golang) [1]. The example of their nfs v3 implementation disables locking. Still pretty cool to see all the ways the rust ecosystem is empowering stuff like this. I’m kinda surprised someone hasn’t integrated the buildbarn nfs v4 stuff into docker/podman - the virtiofs stuff is pretty bad on osx and the buildbarn nfs 4.0 stuff is a big improvement over nfs v3. Anyhow I digress. Can’t wait to take it for a spin. [0] https://github.com/Barre/zerofs_nfsserve [1] https://github.com/buildbarn/bb-remote-execution/tree/master...

Though it obviously depends on the case at hand, I sort of agree with this. For a distributed build cluster that I maintain (Buildbarn, https://github.com/buildbarn/bb-remote-execution/), I also had to implement a scheduler process that would queue compilation/test actions, so that they can be executed on workers later on. Initially I looked into using some conventional queueing system, but eventually settled on implementing my own as part of the scheduler process. So far I'm really happy with this choice, as it has allowed me to implement the following features, and more: - In-flight deduplication of identical compilation actions. If identical actions are scheduled with different priorities, the highest priority is used. - Multi-level scheduling fairness between groups, users in a group, builds run by the same user, etc.. The fairness co

> I don't understand the comment about leaking memory, nfs file handles don't have to be persistent. As long as the file the NFSv4 file handle refers to is still usable (i.e., linked into the file system), the NFSv4 file handle must remain usable. Note that this is not a universal requirement, but at least one that the macOS NFSv4 client enforces. It only implements FH4_PERSISTENT. This doesn't seem to be documented explicitly, but is somewhat revealed by this printf(): https://github.com/apple/darwin-xnu/blob/main/bsd/nfs/nfs_vf... > Besides, how many FUSE filesystems implement FORGET? Any file system that wants to remove files in the background (meaning: not by calling unlink() through the FUSE mount) must likely do proper refcounting on such files, and provide an implementation of FORGET. For example, a file system that can give inform

I was going to raise that point exactly. As someone who spends an awful amount of time using FUSE, my recommendation is to only use it in cases where the software that interacts with the file system isn't easily changeable. For example, for Buildbarn which I maintain (https://github.com/buildbarn/bb-remote-execution), I need to use it. It's infeasible to change arbitrary compilers and tests to all interact with a network distributed build cache. Designing the FUSE file system was a pretty heavy investment though, as you really need to be POSIXly correct to make it all work. The quality of implementations of FUSE also varies between OSes and their versions. macFUSE, for example, is quite different from Linux FUSE. Given that SQLite already has all of the hooks in place, I would strongly recommend using those. In addition to increasing port

Buildbarn, a build cluster implementation for Bazel that I maintain, can also run build actions (compilation steps, unit tests) in a FUSE file system. Though the primary motivator for this is that it reduces the time to construct a build action's file system to nearly instant, it has the advantage that I can also do things similar to disorderfs. Shuffling directory listings is actually something that I also added. Pretty useful! https://github.com/buildbarn/bb-remote-execution/blob/eb1150...