proper

> There are generally two main categories of tests. There are unit tests [and] Integration tests Later on there is a mention of fuzzing. However, there is something better called property tests. Where the on failure there is automatic "shrinking" of input complexity to find a minimally failing test scenario. For Erlang and Elixir I recommend https://propertesting.com/ book and the PropEr test framework https://github.com/proper-testing/proper. As an example, here is the use of an advanced feature called targeted property based testing: https://proper-testing.github.io/tutorials/PropEr_testing_wi... used to test a labyrinth. EDIT: as spockz mentioned below, the original idea was developed by QuviQ as QuickCheck http://www.quviq.com/products/erlang-quickcheck/. It's a commercial offering but there is a QuickCheck mini offered as a free dow

The inputs shouldn’t be “random”, but the idea is that you should provide an input distribution that’s roughly representative of your expected distribution of real-world values (+ some perturbation to find edge cases). As for frameworks in the languages you listed... Ruby and Python: https://github.com/HypothesisWorks/hypothesis Elixir and Erlang: https://github.com/proper-testing/proper Node and JS: https://jsverify.github.io/ As for real world use-cases, imagine you’re writing a program that accepts timestamps as input and has to implement branching, requirements-defined business logic based off of them. When you’re writing your unit tests you can use the requirements to select timestamps that are “known good” and “known bad”, but it’s hard to explore this state space on your own. Same thing goes for handling unexpected inputs to certai

We have a fair bit of property tests in KAZOO now, using PropEr[0], an open source implementation in Erlang. Fred (of Learn You Some Erlang) wrote a great book/site[1] on property testing in Erlang and Elixir, well worth the price of the book. I find property testing is a muscle; the more time I can spend writing them, the "better" I get at writing useful tests. But the muscle atrophies quickly for me, so having the existing tests helps speed up getting into the mindset again. I think the model checking (proper_statem in the PropEr code) is the real winner though. We can model a part of our system, write state transitions and then test those against both the model and a running system and compare our results. Any discrepancy points to either the model or the system being wrong (which is its own joy/pain to figure out). [0] https://github.