coyote

Always good to see more examples of this sort of project; making it straightforward to test how systems behave with unreliable networks [1] should make building resilient systems easier. I came across Microsoft's Coyote project [2] a while back, which seems similar to this, though for C#. Has anyone here tried that out? If so, what have your experiences with it been like? [1] Also known as any network ever. [2] https://github.com/microsoft/coyote

Microsoft has a library/tool called Coyote* that helps with testing distributed systems; you can write tests/specifications, Coyote will systematically explore nondeterminism in your system and check if your tests still pass. If there's a failure, it'll show the sequence of events that led to the failing test. I started a project to implement Raft with a KV-store on top, similar to the article, meaning to use Coyote to test it; I didn't get that far before losing interest, though. It's reassuring to read that it took Phil several months to write the code in the post, it's good to know that this is a decidedly nontrivial problem. * https://github.com/microsoft/coyote